FiveTech Support Forums

FiveWin / Harbour / xBase community
Board index FiveWin for Harbour/xHarbour access to a supplier web site under program control
Posts: 2706
Joined: Fri Oct 07, 2005 01:50 PM
access to a supplier web site under program control
Posted: Fri Apr 01, 2011 05:07 PM

To All

I have a request from a Customer who is a small engine parts dealer that routinely logs in to multiple suppliers web sites and checks availability of specific part numbers. He has asked if it is possible to create a program that will access and login to multiple suppliers web sites in one program and retrieve parts availibility, his cost information and display a grid of each suppliers information so he can choose the best price.

In reviewing the requirements and looking at various Vendor web sites it appears that each supplier has a specific login, userid to access their parts information. What I know I will need is login credentials and the vendors IP address for each site .. with those three variables, how would I connect to a suppliers site under (x)Harbour\FWH control ( sockets perhaps ) ?

I realize this is a vague description of the process, and I will probably have to look at their web page to track the html?? .. I was just curious if anyone has actually remotely logged into a website, passing login parameters and making a connection using (x)Harbor and FWH ?

Would be grateful for any suggestions.

Thanks
Rick Lipkin

Posts: 603
Joined: Sun May 04, 2008 08:44 PM
Re: access to a supplier web site under program control
Posted: Fri Apr 01, 2011 07:58 PM

Search on google by: curllib
make download of curllib for DOS and look samples inside of file downloaded. using he you can execute operation on website request values etc...

Maybe it can help you.

:wink:

Posts: 4840
Joined: Fri Nov 18, 2005 04:52 PM
Re: access to a supplier web site under program control
Posted: Sun Apr 03, 2011 05:47 PM

Rick,

Try searching this forum for "login." This thread is particularly interesting:

viewtopic.php?f=3t=12514start=0hilit=https+patrick

I have not tried any of this so I can't offer any specific help.

Regards,
James

&&&

FWH 18.05/xHarbour 1.2.3/BCC7/Windows 10
Posts: 2706
Joined: Fri Oct 07, 2005 01:50 PM
Re: access to a supplier web site under program control
Posted: Mon Apr 04, 2011 01:14 PM

James

Thank you for your help .. I see there is a userid and login test code .. I will have to research this code further.

I have a meeting this week with our client and will hopefully get some examples of his vendor web sites.

Appreciate your help !

Rick

Posts: 2706
Joined: Fri Oct 07, 2005 01:50 PM
Re: access to a supplier web site under program control
Posted: Tue Apr 05, 2011 04:47 PM

To All

Reviewing the topic James pointed out .. it has led me to this simple active x code :

include "fivewin.ch"

Function Main()
Local oWnd,oActiveX

DEFINE WINDOW oWnd TITLE "Test"

oActiveX = TActiveX():New( oWnd, "Shell.Explorer" )

oWnd:oClient := oActiveX // To fill the entire window surface

oActiveX:Do( "Navigate2", "https://www.dixiesales.com/login-us.aspx" )

ACTIVATE WINDOW oWnd MAXIMIZED

return(nil)

The above code takes me to where I want to be but I need to be able to pass a dealer number to the login as a parameter.

Eventually I want to be able to 'crawl' multiple vendor sites, login and pass a stock number and get availability and dealor cost.

Any ideas would be appreciated.

Thanks
Rick Lipkin

Posts: 4840
Joined: Fri Nov 18, 2005 04:52 PM
Re: access to a supplier web site under program control
Posted: Tue Apr 05, 2011 05:42 PM
Rick,

I think you need something like the code below. I could not get it working with my version of FWH (I think it is 10.06). The activeX object never gets instantiated properly.

You will need to change the 1000 to whatever Dealer number you need.

James

Code (fw): Select all Collapse
/*
Purpose: Test auto login to website
*/


#include "fivewin.ch"

function Main()

   local oWnd, oActiveX

msgInfo() // this shows

   DEFINE WINDOW oWnd

   @ 0, 0 ACTIVEX oActiveX PROGID "Shell.Explorer" OF oWnd

msgInfo( valtype( oActiveX ) )  // This never shows

   oWnd:oClient = oActiveX


msginfo()

   ACTIVATE WINDOW oWnd //;
      ON INIT oActiveX:Do( "Navigate2",;
                           "https://www.dixiesales.com/login-us.aspx",,,;
                           GetPostData( "Dealer Number=1000" ),;
                           "Content-Type: application/x-www-form-urlencoded" + CRLF )

return nil

//-----------------------------------------------------------------------------------------//

#pragma BEGINDUMP

#include <hbapi.h>
#include <windows.h>

HRESULT hb_oleVariantToItem( PHB_ITEM pItem, VARIANT * pVariant );

HB_FUNC( GETPOSTDATA )
{
   VARIANT vPostData = {0};
   LPSAFEARRAY psa;
   LPCTSTR cszPostData = hb_parc( 1 );
   UINT cElems = lstrlen( cszPostData );
   LPSTR pPostData;

   VariantInit( &vPostData );

   psa = SafeArrayCreateVector( VT_UI1, 0, cElems );
   if( ! psa )
   {
      hb_retnl( E_OUTOFMEMORY );
      return;
   }

   SafeArrayAccessData( psa, ( LPVOID * ) &pPostData );
   memcpy( pPostData, cszPostData, cElems );
   SafeArrayUnaccessData( psa );

   V_VT( &vPostData ) = VT_ARRAY | VT_UI1;
   V_ARRAY( &vPostData ) = psa;

   hb_oleVariantToItem( hb_param( -1, HB_IT_ANY ), &vPostData );
}

#pragma ENDDUMP

//end
FWH 18.05/xHarbour 1.2.3/BCC7/Windows 10
Posts: 2706
Joined: Fri Oct 07, 2005 01:50 PM
Re: access to a supplier web site under program control
Posted: Tue Apr 05, 2011 07:54 PM

James

I think this is more complicated that it appears ... I got your code modifications to run .. but I have no idea what the field name is for the login .. I looked at the html code from the page and I need to try this code on a site I have some login credentials on ..

Thank you !

Rick

//---------------------

include "FiveWin.ch"

function Main()

local oWnd, oActiveX

DEFINE WINDOW oWnd

@ 0, 0 ACTIVEX oActiveX PROGID "Shell.Explorer" OF oWnd

oWnd:oClient = oActiveX

ACTIVATE WINDOW oWnd ;
ON INIT oActiveX:Do( "Navigate2",;
"https://www.dixiesales.com/login-us.aspx",,,;
GetPostData( "name=1000" ),;
"Content-Type: application/x-www-form-urlencoded" + CRLF )
return nil

//---------------------------------------------------------------------

pragma BEGINDUMP

include <hbapi.h>

include <windows.h>

HRESULT hb_oleVariantToItem( PHB_ITEM pItem, VARIANT * pVariant );

HB_FUNC( GETPOSTDATA )
{
VARIANT vPostData = {0};
LPSAFEARRAY psa;
LPCTSTR cszPostData = hb_parc( 1 );
UINT cElems = lstrlen( cszPostData );
LPSTR pPostData;

VariantInit( &vPostData );

psa = SafeArrayCreateVector( VT_UI1, 0, cElems );
if( ! psa )
{
hb_retnl( E_OUTOFMEMORY );
return;
}

SafeArrayAccessData( psa, ( LPVOID * ) &pPostData );
memcpy( pPostData, cszPostData, cElems );
SafeArrayUnaccessData( psa );

V_VT( &vPostData ) = VT_ARRAY | VT_UI1;
V_ARRAY( &vPostData ) = psa;

hb_oleVariantToItem( hb_param( -1, HB_IT_ANY ), &vPostData );
}

pragma ENDDUMP

Posts: 2706
Joined: Fri Oct 07, 2005 01:50 PM
Re: access to a supplier web site under program control
Posted: Tue Apr 05, 2011 09:20 PM

Laiton

I did some research into 'curl' ... and by their own definition:

curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP,
FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP,
SFTP, SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user interaction.The 'curl' instructions look interesting but I do not know if this program can be used to just open a url and pass parameters to 'login' to the site ?

Do you have an example you might share with me using 'curl' that I can test .. I do not want to send files ( ftp ..etc ) I just want to be able to login to a site under program control.

Thanks
RIck

Posts: 603
Joined: Sun May 04, 2008 08:44 PM
Re: access to a supplier web site under program control
Posted: Wed Apr 06, 2011 04:58 AM
Rick,

http://www.fiveweb.com.br/curl-test/curl-test.rar

Note I access this url with Curl.exe make login and return me result:
http://www.fiveweb.com.br/curl-test/


( I don't know if is it that you wanna, more please test. )

:-)
Posts: 2706
Joined: Fri Oct 07, 2005 01:50 PM
Re: access to a supplier web site under program control
Posted: Wed Apr 06, 2011 01:48 PM

Lailton

WOW .. I ran your demo and looked at your code .. this certainly logs into your site .. I will do some more testing with your example ..

I appreciate your help and quick response !

Rick

Posts: 4840
Joined: Fri Nov 18, 2005 04:52 PM
Re: access to a supplier web site under program control
Posted: Wed Apr 06, 2011 03:18 PM

Rick,

After rereading your original post, it seems that logging in is only the tip of the iceberg. What you really need is for the website to provide you with a XML file containing the data you need. I believe this was originally done with SOAP which Microsoft has since abandoned, but there must be a replacement technology. This probably is worth some Googling. You might also search this forum for SOAP and XML.

Regards,
James

FWH 18.05/xHarbour 1.2.3/BCC7/Windows 10
Posts: 603
Joined: Sun May 04, 2008 08:44 PM
Re: access to a supplier web site under program control
Posted: Wed Apr 06, 2011 03:45 PM

Good Rick,

If you wanna to get xml file really you can use soap kit of microsoft ( for webservice ), work fine.

more if is only to login you can use curllib.

Good Luck :D

Posts: 2706
Joined: Fri Oct 07, 2005 01:50 PM
Re: access to a supplier web site under program control
Posted: Wed Apr 06, 2011 04:17 PM

Laiton and James ..

Definitly the tip of the iceburg .. and I will need to get with one of the vendors to see how they query up a stock number so I can tap into that process ..

Really appreciate the advice you both have shared !

Rick

Posts: 2706
Joined: Fri Oct 07, 2005 01:50 PM
Re: access to a supplier web site under program control
Posted: Wed Apr 06, 2011 05:26 PM

Lailton

What is the trigger ( submit ) input to get to the next page ? I looked at your Receive.html and reviewed the source ..

Your code evals to the connect logic and returns the html of the page .. but I do not exactically know what 'curl' is doing .. or how to get past a login page once the connection is successful ?

Hope that made sense ??

Thanks
Rick

Posts: 3022
Joined: Fri Oct 07, 2005 01:45 PM
Re: access to a supplier web site under program control
Posted: Wed Apr 06, 2011 07:24 PM

Rich,

You are trying to re-create the wheel ! Most parts suppliers already use software that makes the data available to their clients ... but there is a monthly access fee from the "source" company. They probably can't let you access the data because of the licensing agreement they have with the vendor of that product. You can program to it, but your upfront cost to be "licensed" will be high.

Before the client sets you on a path that costs you a lot, you may want to investigate this more closely with the vendors.

Tim

Tim Stone
http://www.MasterLinkSoftware.com
http://www.autoshopwriter.com
timstone@masterlinksoftware.com
Using: FWH 23.10 with Harbour 3.2.0 / Microsoft Visual Studio Community 2022-24 32/64 bit