FiveTech Support Forums

FiveWin / Harbour / xBase community
Board index FiveWin for Harbour/xHarbour SubNtx
Posts: 44158
Joined: Thu Oct 06, 2005 05:47 PM
SubNtx
Posted: Tue Sep 09, 2008 12:21 PM

Is there any SubNtx and Clipper old user here ?

We are curious to do some speed tests, thanks :-)

regards, saludos

Antonio Linares
www.fivetechsoft.com
Posts: 44158
Joined: Thu Oct 06, 2005 05:47 PM
SubNtx
Posted: Tue Sep 09, 2008 02:38 PM

Please review this topic for the speed tests that we are doing:

http://forums.fivetechsoft.com/viewtopic.php?t=12620

regards, saludos

Antonio Linares
www.fivetechsoft.com
Posts: 199
Joined: Tue Apr 22, 2008 09:54 AM
SubNtx
Posted: Fri Sep 12, 2008 10:39 AM

Antonio,

Very high speed...

Posts: 44158
Joined: Thu Oct 06, 2005 05:47 PM
SubNtx
Posted: Wed Sep 17, 2008 10:36 AM
We have adapted Cesar A. Gil's source code to use it and test it from Harbour and xHarbour:

Here it is the first version. Please notice that we supply it a codeblock so we can do what we want with each NTX key. Next step is to create a new NTX file with the keys that match a certain condition:

test.prg
#include "FiveWin.ch"

FIELD First, Last

function Main()

   USE Customer VIA "DBFNTX"

   INDEX ON First+Last TO Customer
   SET INDEX TO
   
   SubNtx( "customer.ntx", { | nRecNo, cKey | Test( nRecNo, cKey ) } )

return nil

function Test( nRecNo, cKey ) // In this test we show the first 10 keys only

   static nTimes := 1
   
   if nTimes < 11 
      MsgInfo( cKey, "Key: " + AllTrim( Str( nTimes++ ) ) + ;
               " --> RecNo: " + AllTrim( Str( nRecNo ) ) )
   endif
   
return nil      

#pragma BEGINDUMP

#include <hbapi.h>
#include <hbapifs.h>
#include <hbvm.h>

#define MAX_KEY  256
#define BUF_SIZE 1024

typedef struct 
{
	unsigned short int     type;
	unsigned short int     version;
	long     root;
	long     next_page;
	unsigned short int     item_size;
	unsigned short int     key_size;
	unsigned short int     key_dec;
	unsigned short int     max_item;
	unsigned short int     half_page;
	char     key_expr[ MAX_KEY ];
	char     unique;
} NTX_HEADER;

typedef struct
{
	long page;
	long rec_no;
	char key[ 1 ];
} NTX_ITEM;

typedef struct
{
	unsigned short item_count;
	unsigned short item_offset[ 1 ];
} NTX_BUFFER;

static void ReadPage( int hFileIn, long page_offset, PHB_ITEM pCodeBlock, unsigned short int iKeySize )
{
   char ntxPage[ BUF_SIZE ];
	 NTX_ITEM * pNtxItem;
	 NTX_BUFFER * pNtxBuffer;
   int i;
   
   hb_fsSeek( hFileIn, page_offset, FS_SET );
   
	 if( hb_fsRead( hFileIn, ntxPage, BUF_SIZE ) != BUF_SIZE ) 
	    return;
	    
	 pNtxBuffer = ( NTX_BUFFER * ) ntxPage;   
	    
	 for( i = 0; i < pNtxBuffer->item_count; i ++ ) 
	 {
	    pNtxItem = ( NTX_ITEM * ) ( ntxPage + pNtxBuffer->item_offset[ i ] );

 	    if( pNtxItem->page )
		     ReadPage( hFileIn, pNtxItem->page, pCodeBlock, iKeySize );
		     
		  hb_vmPushSymbol( &hb_symEval );
		  hb_vmPush( pCodeBlock );
		  hb_vmPushLong( pNtxItem->rec_no );
		  hb_vmPushString( ( char * ) &pNtxItem->key, iKeySize );
		  hb_vmFunction( 2 );
   }   

	 pNtxItem = ( NTX_ITEM * ) ( ntxPage + pNtxBuffer->item_offset[ pNtxBuffer->item_count ] );
	 
	 if( pNtxItem->page )
	    ReadPage( hFileIn, pNtxItem->page, pCodeBlock, iKeySize );
}

HB_FUNC( SUBNTX )
{
   NTX_HEADER ntx_header;   
   int hFileIn = hb_fsOpen( hb_parc( 1 ), FO_READ );
   PHB_ITEM pCodeBlock = hb_param( 2, HB_IT_BLOCK );

   if( hFileIn != -1 )
   {
      hb_fsRead( hFileIn, ( char * ) &ntx_header, sizeof( NTX_HEADER ) );
      
      ReadPage( hFileIn, ntx_header.root, pCodeBlock, ntx_header.key_size );

      hb_fsClose( hFileIn );    
   }  
}

#pragma ENDDUMP
regards, saludos

Antonio Linares
www.fivetechsoft.com
Posts: 842
Joined: Mon Oct 10, 2005 01:29 PM
SubNtx
Posted: Wed Sep 17, 2008 01:17 PM

Hello Antonio

a stupid question .
This works with DBFCDX to ?

MAurizio

Posts: 44158
Joined: Thu Oct 06, 2005 05:47 PM
SubNtx
Posted: Wed Sep 17, 2008 01:27 PM

Maurizio,

No. Actually it is only for NTX.

But we are curious to check if it could be implemented for CDX too :-)

regards, saludos

Antonio Linares
www.fivetechsoft.com
Posts: 824
Joined: Thu Oct 13, 2005 07:39 AM
SubNtx
Posted: Thu Sep 18, 2008 07:16 AM

Antonio,

just to understand what you are doing.... (so maybe another stupid question :wink: )

What is SubNtx and what are the advantages using this function ?

kind regards

Stefan
Posts: 44158
Joined: Thu Oct 06, 2005 05:47 PM
SubNtx
Posted: Thu Sep 18, 2008 07:27 AM

Stefan,

SubNtx was a very popular Clipper tool that allows to perform incredibly fast queries in DBFs, providing the results in few seconds.

We have done some speed tests doing queries on large DBFs and SubNtx takes 3,5 seconds where Harbour/xHarbour takes almost a minute.

SubNtx basically opens the index itself and makes the search inside the index. It does not uses the RDD internal functions, neither the Harbour/xHarbour virtual machine. Just pure C at the lowest level. The results are really impressive :-)

For example: Imagine that you have a DBF and you want to make a query for different values in different fields. Here is where SubNtx is clearly the fastest option.

regards, saludos

Antonio Linares
www.fivetechsoft.com
Posts: 824
Joined: Thu Oct 13, 2005 07:39 AM
SubNtx
Posted: Thu Sep 18, 2008 07:31 AM

Antonio,

that sounds very promising, thanks for the info

kind regards

Stefan

Continue the discussion