Transformer Tutorial

The TTransformer component implements the full transformer architecture described in the seminal paper "Attention Is All You Need" (Vaswani et al., 2017). It allows you to load pre-trained model weights, run inference, fine-tune on custom data, and even train small transformers from scratch — all from Harbour xBase code.

Sample Projects

Transformer Architecture

Properties

Events

Example 1: Load a Pre-Trained Model

#include "hbbuilder.ch"

function Main()

   local oTransformer, cOutput

   DEFINE TRANSFORMER oTransformer ;
      MODEL "models/tinyllama-1.1b-chat.Q4_K_M.gguf" ;
      CONTEXT 2048 ;
      GPU_LAYERS 0

   if .not. oTransformer:lLoaded
      ? "Failed to load model:", oTransformer:cError
      return nil
   endif

   // Generate text from a prompt
   cOutput := oTransformer:Generate( ;
      "Once upon a time", ;  // prompt
      128 )                  // max tokens

   ? cOutput

return nil

Example 2: Streaming Token Generation

static function StreamExample()

   local oTransformer

   DEFINE TRANSFORMER oTransformer ;
      MODEL "models/phi3-mini.Q4_K_M.gguf"

   // Hook into the OnGenerate event for streaming
   oTransformer:OnGenerate := { |cToken, nStep| ;
      QOut( cToken ) }

   oTransformer:GenerateStream( ;
      "Explain the transformer architecture", ;
      256 )

return nil

Example 3: Attention Visualization

static function AttentionVis()

   local oTransformer, oForm, oChart
   local aAttentionMaps := {}

   DEFINE TRANSFORMER oTransformer ;
      MODEL "models/tinyllama-1.1b.Q4_K_M.gguf"

   // Capture attention weights via OnAttention event
   oTransformer:OnAttention := ;
      { |aW, nL, nH| AAdd( aAttentionMaps, ;
         { nL, nH, aW } ) }

   // Run a forward pass
   oTransformer:Forward( "The cat sat on the mat" )

   // Now aAttentionMaps contains weight matrices
   // for each layer and head - render as heatmap
   RenderAttentionHeatmap( aAttentionMaps )

return nil

Example 4: Training from Scratch

static function TrainExample()

   local oTransformer
   local aInputs  := { "hello", "world", "test" }
   local aTargets := { "hello", "world", "test" }  // copy task

   DEFINE TRANSFORMER oTransformer ;
      LAYERS 2 ;
      HEADS 4 ;
      EMBED_DIM 128 ;
      FF_DIM 256 ;
      VOCAB_SIZE 1000 ;
      MAX_SEQ_LEN 32 ;
      DROPOUT 0.1

   // Monitor training progress
   oTransformer:OnTrainStep := ;
      { |nStep, nLoss, nBatch| ;
         QOut( "Step:", nStep, "Loss:", nLoss ) }

   oTransformer:OnLoss := ;
      { |nLoss| ;
         if nLoss < 0.01 ;
            QOut( "Converged!" ) ;
         endif }

   // Train for 1000 steps
   oTransformer:Train( aInputs, aTargets, ;
      1000,  // steps
      0.001 ) // learning rate

return nil

Architecture Details

Attention Mechanism

Each attention head computes the standard scaled dot-product attention:

Attention(Q, K, V) = softmax(Q * K^T / sqrt(d_k)) * V

The multi-head mechanism runs nHeads parallel attention computations and concatenates the results, then applies a linear projection.

Causal Masking

When lCausal is .T. (the default), the transformer applies a causal (triangular) mask that prevents each position from attending to future positions. This is essential for autoregressive text generation. Set lCausal := .F. for bidirectional tasks like classification or masked language modeling.

Feed-Forward Network

Each layer contains a position-wise feed-forward network:

FFN(x) = max(0, x * W1 + b1) * W2 + b2

where the inner dimension is nFFDim (typically 4x the embedding dimension).