topic-33746

Posts: 9020

Joined: Thu Oct 06, 2005 08:17 PM

Re: Automated download of pictures
Posted: Thu Mar 09, 2017 10:57 AM

nageswaragunupudi wrote:Thank you.
GETURLTOFILE() is working with http: but not https:

This is not true. It would work if tipssl were linked (and opensll DLL's were available).

nageswaragunupudi wrote:URLDOWNLOADTOFILE() is working well with all and is also much faster than GETURLTOFILE().

I was not aware the URLDOWNLOADTOFILE() is much faster than GETURLTOFILE(). Are you made any real test? Can you share the timing result?

nageswaragunupudi wrote:The actual reason for my testing is to find a faster alternative to the existing function WebPageContents(). Our requirement is to directly read the contents of the URL into memory.

I use this (but it needs opensll as well):

Code (fw): Select all Collapse

FUNCTION GETURL( cUrl )

    LOCAL cHtm := ""

    LOCAL oCli

    TRY
        oCli = TIPClientHttp():New( cUrl )

        IF EMPTY( oCli ) THEN BREAK

        oCli:nConnTimeout = -1

        IF !oCli:Open() THEN BREAK

        cHtm = oCli:ReadAll()

        IF EMPTY( oCli:cReply ) .OR. !( "OK" $ UPPER( oCli:cReply ) )
            cHtm = ""
        ENDIF
    CATCH
    END

    IF !EMPTY( oCli ) THEN oCli:Close()

    RETURN cHtm

EMG

Enrico Maria Giordano

e.m.giordano@emagsoftware.it
http://www.emagsoftware.it
http://www.emagsoftware.it/emgmusic
http://www.emagsoftware.it/spectrum
http://www.emagsoftware.it/tbosg

nageswaragunupudi

Posts: 10733

Joined: Sun Nov 19, 2006 05:22 AM

Re: Automated download of pictures
Posted: Thu Mar 09, 2017 01:25 PM

I was not aware the URLDOWNLOADTOFILE() is much faster than GETURLTOFILE(). Are you made any real test? Can you share the timing result?

I used the Tajmahal.jpg ( http not https )

GETURLTOFILE() --> 3.1 to 3.3 secs
Both
URLDOWNLOADTOFILE() and
MEMOWRIT( cFile, webpagecontents( curl ) ) --> 0.06 to 0.09 secs

Code (fw): Select all Collapse

function testweb4()

   local cURL := "http://cdn.history.com/sites/2/2015/04/hith-eiffel-tower-iStock_000016468972Large.jpg"
   local cFile := "test.jpg"
   local nSecs

   FERASE( cFile )
   URLDOWNLOADTOFILE( 0, cURL, cFile ) // Ignore time taken for the first time

   nSecs := SECONDS()
   GETURLTOFILE( cURL, cFile )
   ? SECONDS() - nSecs, File( cFile ) // --> 3.23

   nSecs := SECONDS()
   URLDOWNLOADTOFILE( 0, cURL, cFile )
   ? SECONDS() - nSecs, File( cFile ) // --> 0.07

   nSecs := SECONDS()
   MEMOWRIT( cFile, webpagecontents( curl ) )
   ? SECONDS() - nSecs, File( cFile )  // --> 0.07

   if File( cFile )
      XIMAGE( cFile )
   endif

return nil

Regards

G. N. Rao.

Hyderabad, India

Enrico Maria Giordano

Posts: 9020

Joined: Thu Oct 06, 2005 08:17 PM

Re: Automated download of pictures
Posted: Thu Mar 09, 2017 02:46 PM

That's only a cache effect. Try calling this before download:

Code (fw): Select all Collapse

DLL FUNCTION DELETEURLCACHEENTRY( cUrl AS LPSTR ) AS BOOL;
    PASCAL FROM "DeleteUrlCacheEntryA" LIB "wininet.dll"

EMG

Enrico Maria Giordano

e.m.giordano@emagsoftware.it
http://www.emagsoftware.it
http://www.emagsoftware.it/emgmusic
http://www.emagsoftware.it/spectrum
http://www.emagsoftware.it/tbosg

nageswaragunupudi

Posts: 10733

Joined: Sun Nov 19, 2006 05:22 AM

Re: Automated download of pictures
Posted: Fri Mar 10, 2017 05:32 PM

Mr EMG

You are right.
Without the benefit of cache, GETURLTOFILE() is faster than the other two.
Another observation is that GETURLTOFILE() does not take advantage of cache and other two take advantage of cache if available

Regards

G. N. Rao.

Hyderabad, India

Enrico Maria Giordano

Posts: 9020

Joined: Thu Oct 06, 2005 08:17 PM

Re: Automated download of pictures
Posted: Fri Mar 10, 2017 06:09 PM

nageswaragunupudi wrote:Mr EMG

You are right.
Without the benefit of cache, GETURLTOFILE() is faster than the other two.
Another observation is that GETURLTOFILE() does not take advantage of cache and other two take advantage of cache if available

That's true. But use of cache means that the file is not really downloaded, even if it's changed. So use it with caution.

EMG

Enrico Maria Giordano

e.m.giordano@emagsoftware.it
http://www.emagsoftware.it
http://www.emagsoftware.it/emgmusic
http://www.emagsoftware.it/spectrum
http://www.emagsoftware.it/tbosg

nageswaragunupudi

Posts: 10733

Joined: Sun Nov 19, 2006 05:22 AM

Re: Automated download of pictures
Posted: Sat Mar 11, 2017 03:27 AM

That's true. But use of cache means that the file is not really downloaded, even if it's changed. So use it with caution.

Yes. Thanks

Regards

G. N. Rao.

Hyderabad, India

FiveTech Support Forums

Continue the discussion