FiveTech Support Forums

FiveWin / Harbour / xBase community
Board index FiveWin for Harbour/xHarbour Text file processing.
Posts: 723
Joined: Tue Sep 04, 2007 08:45 AM
Text file processing.
Posted: Mon Nov 02, 2009 04:47 PM

What class / command / function can I use to process (READ / WRITE ) Text files ? Are there any samples ? Thank you.

Posts: 6983
Joined: Fri Oct 07, 2005 07:07 PM
Re: Text file processing.
Posted: Mon Nov 02, 2009 05:29 PM

I use:
memowrit( "info.txt",cText)
Memoread(cAppPath + "\ini\update.txt")

Best regards,
Otto

Posts: 4043
Joined: Wed Dec 19, 2007 06:40 PM
Re: Text file processing.
Posted: Mon Nov 02, 2009 05:41 PM
The way to write a Textfile ( PRG ) with formatted Textlines , read with a Editor, compile and run the EXE-file :
A Example, to write the Source for a Window with included Buttons.
I use it in all of my Tools, to write the Source and create a EXE-File.

Code (fw): Select all Collapse
FUNCTION MAKE_EXE()

cNewFile := c_path + "\PROJECT.prg"
cNewFile1 := c_path + "\PROJECT.EXE"

DELETE FILE  "&cNewFile"
DELETE FILE   "&cNewFile1"

NEW_SOURCE() // Creates the Textfile ( PRG )

IF FILE ( c_path + "\PROJECT.PRG" )
    // File exists open with Editor
    WAITRUN( "Wordpad.exe " + c_path + "\PROJECT.prg" ) // copy from .PRG
    // Close Editor and create EXE-File
    IF MsgYesNo( "Do you want to Create / Run : PROJECT.exe ?","PROJECT")   
        WAITRUN( "GO.BAT"  ) // the Link-File, to create the EXE
        IF FILE ( c_path + "\PROJECT.EXE" )
            WINEXEC( "PROJECT.EXE" )
        ELSE
            MsgAlert( "The file : PROJECT.EXE " + CRLF + ;
                  "is not created !","Error" )
        ENDIF
    ENDIF
ELSE
    MsgAlert( "The File : " + CRLF + ;
            "PROJECT.PRG" + CRLF + ;
            "is not created !", "Error" )
ENDIF
SYSREFRESH()

RETURN NIL

// ------- Create the Text-File ( PRG ) --------------------------------

// A Example, how to include Var's :
// oText:Add('Gradient( hDC, { 0, 0, oDlg:nHeight, oDlg:nWidth }, ' + ALLTRIM( STR(W_COLOR1)) + ", " + ALLTRIM( STR(W_COLOR2)) + ', .F. )')
// Including a Textstring You must use ' and " it looks like : oText:Add('Adding Text-String : ' + "Test" + ' LineEnd' )

FUNCTION NEW_SOURCE()

cNewFile := c_path + "\PROJECT.PRG" 
IF FILE ("&cNewFile")
    DELETE FILE  "&cNewFile"
ENDIF

oText := TTxtFile():New( "&cNewFile" )

IF oText:Open()
    oText:Add('#include "FiveWin.ch"')
    oText:Add('#include "TTitle.ch"')
    oText:Add('#include "Image.ch"')
    oText:Add("")
    oText:Add("static oWnd, oButtFont" )
    oText:Add("")
    oText:Add("FUNCTION Main()")
    oText:Add('LOCAL oBtn1, oBtn2, hDC, oBrush, oImage')
    oText:Add("LOCAL nWidth := GetSysMetrics(0)")
    oText:Add("LOCAL nHeight := GetSysMetrics(1)")
    oText:Add("")
    oText:Add("SET _3DLOOK ON" )
    oText:Add("SetBalloon( .T. )" ) 
    oText:Add('c_path := CURDRIVE() + ":\" + GETCURDIR()') 
    oText:Add('oProgFont := TFont():New("Arial", ,-14,.F.,.F. , , , ,.F. ) ')
    oText:Add("")
    oText:Add('DEFINE WINDOW  oWnd  TITLE "VTitle Test" ') 
    oText:Add("")
    oText:Add('oButtFont := TFont():New("Arial",0,-16,.F.,.T.,0,0,0,.T. ) ')
    oText:Add('DEFINE IMAGE oImage FILENAME c_path + "\Images\BACKGRD.JPG" ')
    oText:Add("")
    oText:Add("@ nHeight - 160, nWidth - 420 BTNBMP oBtn1 SIZE 200, 60 OF oWnd  2007 ;" )
    oText:Add('FILENAME c_path + "\Images\select.bmp" ;')
    oText:Add("LEFT ;" )
    oText:Add('PROMPT "  &Preview VTitle " ; ')
    oText:Add("FONT oButtFont ;" )
    oText:Add("ACTION Tools(oWnd)" )  
    oText:Add("oBtn1:lTransparent = .t. " )  
    oText:Add('oBtn1:cTooltip :=  { "Open" + CRLF + ;')
    oText:Add('       "the VTitlt-Painter","VTitle-Painter", 1, CLR_BLACK, 14089979 }' )
    oText:Add("")
    oText:Add("@ nHeight - 160, nWidth - 200 BTNBMP oBtn2 SIZE 150, 60 OF oWnd  2007 ;" )
    oText:Add('FILENAME c_path + "\Images\exit.bmp" ;' )
    oText:Add("LEFT ;" )
    oText:Add('PROMPT "  &Exit  " ;' )
    oText:Add("FONT oButtFont ;" )
    oText:Add("ACTION oWnd:End()" )  
    oText:Add("oBtn2:lTransparent = .t." )   
    oText:Add('oBtn2:cTooltip :=  { "Close" + CRLF + ;' )
    oText:Add('        "the VTitle-Painter","Close Window", 1, CLR_BLACK, 14089979 }' )
    oText:Add("")
    oText:Add('ACTIVATE WINDOW oWnd MAXIMIZED ;') 
    oText:Add('ON PAINT ( DRAWBITMAP( hdc, oImage:hbitmap, 0, 0, nWidth, nHeight ) ) ;') 
    oText:Add('VALID MsgYesNo( "Do you want to end?" )')
    oText:Add("")
    oText:Add('RETURN( NIL )') 

    oText:Close()
ENDIF

RETURN(  NIL )


Best Regards
Uwe :-)
Since 1995 ( the first release of FW 1.9 )

i work with FW.

If you have any questions about special functions, maybe i can help.
Posts: 723
Joined: Tue Sep 04, 2007 08:45 AM
Re: Text file processing.
Posted: Mon Nov 02, 2009 06:31 PM

Otto, Ukoenig:

Thank you for your responses. :D What I'm trying to do is process a huge (3.9GB) CSV file, read it line by line and split it in n lines pieces so I can import it to dbf files. I'm doing it because I have not found a single software that is able to import such a file. I think that the dBaseIII format has a 2GB limitation. So I'm importing it in various pieces. Basically I'm reading the line until I find CHR(13) + CHR(10) and then continue with the next. Thank you.

Posts: 6983
Joined: Fri Oct 07, 2005 07:07 PM
Re: Text file processing.
Posted: Mon Nov 02, 2009 08:12 PM
Best regards,
Otto

Please have a look at this threads:
Maybe you can use strtoken.

http://forums.fivetechsupport.com/viewtopic.php?f=3&t=12318&p=61651&hilit=strtoken#p61651

Antonio shows how to read large files:

http://forums.fivetechsupport.com/viewtopic.php?f=3&t=968&p=25897&hilit=search+otto#p25897

Some years ago we helped a company to test several third party tools for such purpouse and finally implemented our own solution to make the fastest search on all fields of a DBF.

We found with great surprise that the solution was to open the DBF as a standard file with FOpen(), read a bunch of bytes in memory and perform a simple At() to locate a string. Once found, you substract the DBF header length, then divide the offset by the record size and you get the record number. At() is an extremelly fast function as it is directly performed by the processor.

These days that we use 32 bits flat memory, I guess there is no need to use a bunch of bytes, so the entire DBF may be loaded in memory doing a MemoRead() of the DBF file, or several bunchs if it is too large, so the code may get simpler.

We compared this way with other available third party tools, and we found that ours was the fastest one
Posts: 581
Joined: Tue Oct 11, 2005 11:28 AM
Re: Text file processing.
Posted: Mon Nov 02, 2009 09:04 PM

Hi Hunter,

Why don't you use the TExcel class or TOleauto class to import this CSV file into various dbf files? Just a suggestion.

Regards,

Kleyber Derick



FWH / xHb / xDevStudio / SQLLIB
Posts: 44158
Joined: Thu Oct 06, 2005 05:47 PM
Re: Text file processing.
Posted: Mon Nov 02, 2009 10:56 PM
Gustavo,

If you need to read line by line, we have found that this is a very fast way to do it, much faster than using MemoLine():
Code (fw): Select all Collapse
function ExtractLine( cText, nFrom )

  local cLine, nAt

  nAt := At( CRLF, SubStr( cText, nFrom ) )

  if nAt > 0
    cLine := Substr( cText, nFrom, nAt - 1 )
    nFrom += nAt + 1
  else
    cLine := Substr( cText, nFrom )
    nFrom := Len( cText ) + 1
  endif

return cLine

nFrom must be supplied by reference and initialized with 1:
Code (fw): Select all Collapse
   local nFrom := 1
   local nLen  := Len( cText )

    While nFrom <= nLen
       cLine  := ExtractLine( cText, @nFrom )
       ...
    end
regards, saludos

Antonio Linares
www.fivetechsoft.com

Continue the discussion