topic-44357

Board index FiveWin for Harbour/xHarbour Help with HB_REALLOC ERROR 9009

Posts: 883

Joined: Tue Oct 11, 2005 11:57 AM

Help with HB_REALLOC ERROR 9009
Posted: Wed Mar 20, 2024 07:38 PM

Hello Fivewinners.
Hola Fivewinners

Need to open 160.000+ xml files and extract some info
Necesito abrir mas de 160.000 archivos xml y sacar informacion

I use tXmlDocument class
Uso la clase TxmlDocument

I have the file names in an Array
Tengo los nombres de los archivos en un arreglo

Code (fw): Select all Collapse

    For X=1 to 160.xxx

        If z=400
           HB_GcAll()
           SysRefresh()
           z:=0
        Endif

        LeeFileXml(hFile[x])
  
    Next
--------------------------------------
--------------------------------------
Function LeeFile Xml(hFile)
Local oXmlDoc := TXmlDocument():New( hFile )
Local oXmlIter := TXmlIterator():New( oXmlDoc:oRoot )
Local Data1:=0
Local Data2:=""

   while .T.
         oTagActual = oXmlIter:Next()
         If oTagActual != nil

            If oTagActual:cName ="END"
              Exit
            Endif

           If oTagActual:cName ="ValorXxx"
              Data1:=Val(oTagActual:cData)
           Endif

           If oTagActual:cName ="ValorYyy"
              Data2:=oTagActual:cData
           Endif

      End

FClose( hFile )
oXmlDoc:=Nil
oXmlIter:=Nil

 oSvr:Insert("detalle",{"NUMERO","DESCRIPCION"},{Data1,Data2})

Return Nil

It does what it is supposed to do, But I get this error each time Between 5.000 and 6.000 processed files
Hace lo que se supone debe hacer, pero tengo cada vez tengo este error entre los 5.000 y 6.000 archivos procesados

Application Internal Error - C:\SQL\xNewAqua\xNewII.Exe
Terminated at: 2024-03-20 16:18:46
Error irrecuperable 9009: hb_xrealloc no puede reubicar la memoria......

It fails in no specific file, just after 5000+ file
No falla en ningun archivo especifico, solo despues de los 5000

Any help will be appreciated
Cualquier ayuda se agradece

HARBOUR, FWH, MSVC, WINDOWS 11, 32GB ram, NVME disk

;-) Ji,ji,ji... buena la cosa... "all you need is code"

http://www.xdata.cl - Desarrollo Inteligente
----------
Asus TUF F15, 32GB Ram, 2 * 1 TB NVME M.2, GTX 1650

Marc Venken

Posts: 1487

Joined: Tue Jun 14, 2016 07:51 AM

Re: Help with HB_REALLOC ERROR 9009
Posted: Wed Mar 20, 2024 08:01 PM

Run out of memory ?

Maybe you can put a part of this code inside the loop en show the result while running. Than you see if there is a memory leak.

Code (fw): Select all Collapse

cErrorLog += "   Hardware memory: " + ;
                    cValToChar( Int( nExtMem() / ( 1024 * 1024 ) ) + 1 ) + " MB ( available: "+;
                    cValToChar (Int (nAvailMem() / (1024 * 1024 ) ) +1 ) + " MB )" + CRLF + CRLF

Marc Venken

Using: FWH 23.08 with Harbour

Adolfo

Posts: 883

Joined: Tue Oct 11, 2005 11:57 AM

Re: Help with HB_REALLOC ERROR 9009
Posted: Wed Mar 20, 2024 08:45 PM

Hi Marc

To Avoid Memory leaks, within the loop I call an external function each time I read a xml file, close it and turn to nil each class instance.

For every 400 files I do a HB_GcAll() and a SysRefresh(), the biggest xml file is just 29 kb

Im going to put your code inside the loop, and show it every 1.000 files, hope it helps to know

Thanks

;-) Ji,ji,ji... buena la cosa... "all you need is code"

http://www.xdata.cl - Desarrollo Inteligente
----------
Asus TUF F15, 32GB Ram, 2 * 1 TB NVME M.2, GTX 1650

Marc Venken

Posts: 1487

Joined: Tue Jun 14, 2016 07:51 AM

Re: Help with HB_REALLOC ERROR 9009
Posted: Wed Mar 20, 2024 08:48 PM

nAvailMem() seems not to be standard.

These seems to be ok.

   cErrorLog += "   Dynamic memory consume:" + CRLF

   cErrorLog += "      Actual  Value: " + Str( MemUsed() ) + " bytes" + CRLF

   cErrorLog += "      Highest Value: " + Str( MemMax() ) + " bytes" + CRLF

Marc Venken

Using: FWH 23.08 with Harbour

Adolfo

Posts: 883

Joined: Tue Oct 11, 2005 11:57 AM

Re: Help with HB_REALLOC ERROR 9009
Posted: Thu Mar 21, 2024 01:03 AM

I found the error

Disable Avast Antivirus, and it works with no problem.

;-) Ji,ji,ji... buena la cosa... "all you need is code"

http://www.xdata.cl - Desarrollo Inteligente
----------
Asus TUF F15, 32GB Ram, 2 * 1 TB NVME M.2, GTX 1650

karinha

Posts: 8515

Joined: Tue Dec 20, 2005 07:36 PM

Re: Help with HB_REALLOC ERROR 9009
Posted: Thu Mar 21, 2024 12:17 PM

Code (fw): Select all Collapse

FUNCTION TU_Func()

   // inicio
   hb_gcAll( .F. )

   FOR X = 1 TO 160.xxx

      SYSREFRESH()

      IF z = 400

         // HB_GcAll()

         // SysRefresh()

         z := 0

      ENDIF

      LeeFileXml( hFile[ x ] )

   NEXT

   // Salida
   hb_gcAll( .T. )

RETURN NIL

FUNCTION LeeFile Xml( hFile )

   LOCAL oXmlDoc := TXmlDocument():New( hFile )
   LOCAL oXmlIter := TXmlIterator():New( oXmlDoc:oRoot )
   LOCAL Data1 := 0
   LOCAL Data2 := ""

   hb_gcAll( .F. )

   WHILE( .T. )

      SYSREFRESH() // aqui

      oTagActual = oXmlIter:Next()

      IF oTagActual != nil

         IF oTagActual:cName = "END"

            EXIT

         ENDIF

         IF oTagActual:cName = "ValorXxx"

            Data1 := Val( oTagActual:cData )

         ENDIF

         IF oTagActual:cName = "ValorYyy"

            Data2 := oTagActual:cData

         ENDIF

      ENDIF

   ENDDO // ??

   FClose( hFile )

   oXmlDoc  := Nil
   oXmlIter := Nil

   oSvr:Insert( "detalle", { "NUMERO", "DESCRIPCION" }, { Data1, Data2 } )

   hb_gcAll( .T. )

RETURN NIL

Regards, saludos.

João Santos - São Paulo - Brasil - Phone: +55(11)95150-7341

FiveTech Support Forums

Continue the discussion