FiveTech Support Forums

FiveWin / Harbour / xBase community
Board index FiveWin for Harbour/xHarbour read a big xml file
Posts: 357
Joined: Thu Nov 02, 2006 06:53 PM
read a big xml file
Posted: Fri Jun 23, 2023 02:35 PM

I have a problem reading xml file size 10 MB

I got memory error

best regards

kajot
Posts: 8515
Joined: Tue Dec 20, 2005 07:36 PM
Posts: 357
Joined: Thu Nov 02, 2006 06:53 PM
Re: read a big xml file
Posted: Sat Jun 24, 2023 06:32 AM

I got error 9600

hb_xgrab can't allocate memory

xml file has 10GB

best regards

KAJOT

best regards

kajot
Posts: 159
Joined: Wed Mar 28, 2007 01:19 PM
Re: read a big xml file
Posted: Sat Jun 24, 2023 08:42 PM

Hi... (sorry... I was out for a while...)

Kajot,

10Gb XML should be an issue even for Chrome or IE(Edge)

Can you post the XML in soma place for download? (or give an URL)

Regards, Euclides

Posts: 357
Joined: Thu Nov 02, 2006 06:53 PM
Re: read a big xml file
Posted: Sun Jun 25, 2023 05:57 AM

How can I post you this xml file ?

best regards

kajot
Posts: 8515
Joined: Tue Dec 20, 2005 07:36 PM
Re: read a big xml file
Posted: Sun Jun 25, 2023 12:34 PM
kajot wrote:How can I post you this xml file ?
https://mega.nz

Regards, saludos.
João Santos - São Paulo - Brasil - Phone: +55(11)95150-7341
Posts: 357
Joined: Thu Nov 02, 2006 06:53 PM
Re: read a big xml file
Posted: Sun Jun 25, 2023 03:17 PM
best regards

kajot
Posts: 8515
Joined: Tue Dec 20, 2005 07:36 PM
Re: read a big xml file
Posted: Sun Jun 25, 2023 03:21 PM
Have you tried like this?
Code (fw): Select all Collapse
   LOCAL vXML, oText, TempText, aText, cXML

   cXML := cDirPleno + "\" + ALLTRIM( "REMESSA\PROTOCOLOS\" )    + ;
           ALLTRIM( ( cAlias )->CHAVE )                          + ;
           "-CCe"+ALLTRIM(STR( ( cAlias )->CODI ) ) + ".xml"

   IF .NOT. FILE( cXML )
   // MENS
   ENDIF

   vXML := cXML

   oText = TTxtFile():New( vXML ) // Read Mode

   oText:nMaxLineLength := 5120

   TempText = oText:ReadLine()
Regards, saludos.
João Santos - São Paulo - Brasil - Phone: +55(11)95150-7341
Posts: 6755
Joined: Wed Feb 15, 2012 08:25 PM
Re: read a big xml file
Posted: Sun Jun 25, 2023 04:34 PM
kajot wrote:https://mega.nz/file/xeFx3SCS#LOZWjmZzyMM1HZ61g7OwNTYYUhUFfmgGhpYocPbbtTk

best regards
kajot
Try
Code (fw): Select all Collapse
#include "FiveWin.ch"

Static aValues  := {}

function Main()

   XmlTree( "JPK_FAKTURA_22-06-2023.xml" )
   XBrowse( aValues )

return nil

//----------------------------------------------------------------------------//

static function XmlTree( cXml )

   local oXmlDoc
   local oRoot
   local cPrompt
   local cData
   local oTree

   TRY
      oXmlDoc  := TXmlDocument():New( cXml )
   CATCH
      ? "TXmlDocument class can not read " + cXml
      return nil
   END
   oRoot    := oXmlDoc:oRoot
   cPrompt  := If( Empty( oRoot:cName ), cFileNoPath( cXml ), oRoot:cName )
   cData    := if( Empty( oRoot:cData ), "Empty Value", oRoot:cData )
   AAdd( aValues, { cPrompt, cData, oRoot:oChild, oRoot:oNext, oRoot:oPrev, oRoot:oParent, oRoot:Path() } )
   if !Hb_IsNil( oRoot:oChild )
      SubTree( oRoot:oChild )
   endif

return nil

static function SubTree( oNode )

   local cPrompt
   local cData
   
   do while oNode != nil
      if !Empty( oNode:aAttributes ) //Valtype( oNode:aAttributes ) = "H"
         HEVal( oNode:aAttributes, { | cKey, cValue | AAdd( aValues, { cKey, cValue, , , , , oNode:Path()} ) } )      
      endif

      cPrompt  := If( Empty( oNode:cName ), "", oNode:cName )
      cData    := if( Empty( oNode:cData ), "Empty Value", oNode:cData )
      AAdd( aValues, { cPrompt, cData, oNode:oChild, oNode:oNext, oNode:oPrev, oNode:oParent, oNode:Path() } )
      if !Hb_IsNil( oNode:oChild )
         SubTree( oNode:oChild )
      endif

      oNode    := oNode:oNext
   enddo

return nil
Cristobal Navarro

Hay dos tipos de personas: las que te hacen perder el tiempo y las que te hacen perder la noción del tiempo

El secreto de la felicidad no está en hacer lo que te gusta, sino en que te guste lo que haces
Posts: 357
Joined: Thu Nov 02, 2006 06:53 PM
Re: read a big xml file
Posted: Sun Jun 25, 2023 05:25 PM

How can I use

oText:ReadLine()

with

TXmlDocument():New( )

KAJOT

best regards

kajot
Posts: 6755
Joined: Wed Feb 15, 2012 08:25 PM
Re: read a big xml file
Posted: Sun Jun 25, 2023 06:14 PM
kajot wrote:How can I use
oText:ReadLine()

with
TXmlDocument():New( )

KAJOT
what do you need to use that for?
Cristobal Navarro

Hay dos tipos de personas: las que te hacen perder el tiempo y las que te hacen perder la noción del tiempo

El secreto de la felicidad no está en hacer lo que te gusta, sino en que te guste lo que haces
Posts: 8515
Joined: Tue Dec 20, 2005 07:36 PM
Re: read a big xml file
Posted: Mon Jun 26, 2023 01:11 PM
Code (fw): Select all Collapse
// C:\FWH\SAMPLES\KAJOTXML.PRG

#include "FiveWin.ch"

FUNCTION Main()

   // LOCAL oText := TTxtFile():New( "leandro.xml" )  // only test
   LOCAL oText := TTxtFile():New( "FAKTURA.xml" )     // KAJOT
   LOCAL n, cText := ""
   LOCAL cImptxt, cTexto, oPrn, oFont, nLin, nLinha

   IF( .NOT. oText:Open() )

      ? "impossible Open"

      oText:Close()

      QUIT

   ENDIF

   // number max. lines.
   oText:nMaxLineLength := 350000  // KAJOT - Look this.

   oText:RecCount()

   // ONLY TEST
   // FOR n = 1 TO 5120

   FOR n = 1 TO 330000   // FACTURA.XML -> 326.642 lines.  KAJOT

     oText:ReadLine()

     cText += oText:ReadLine()

     oText:Skip()

   NEXT

   cTexto := ALLTRIM( cText )

   // ? cTexto

   PRINTER oPrn NAME "Imprimindo um TXT" PREVIEW MODAL

   DEFINE FONT oFont NAME "COURIER NEW" SIZE 0, - 8 OF oPrn

   oPrn:SetPage( 9 )    // A4
   oPrn:SetPortrait()   // Vertical

   PAGE

      nLin := 1

      FOR nLinha = 1 TO MLCOUNT( cTexto, 100 )

         SYSREFRESH()

         oPrn:CmSay( nLin := nLin + .4, 1.5, MEMOLINE( cTexto, 100, nLinha ), oFont )

         IF nLin > 25

            nLin := 1

            ENDPAGE

            PAGE

         ENDIF

      NEXT

      ENDPAGE

   ENDPRINT

   oFont:End()

   oText:Close()

RETURN NIL

// FIN / END
Regards, saludos.
João Santos - São Paulo - Brasil - Phone: +55(11)95150-7341

Continue the discussion