Tensor Logic

Source: source/classes/tensorlogic.prg, source/function/fwtensor.c

FiveWin implements Pedro Domingos' Tensor Logic: The Language of AI (arXiv:2510.12269). Its key insight: a logical rule and an Einstein summation are the same operation. A program is just a set of tensor equations; each equation joins the tensors on its right-hand side (product over shared indices), projects out the indices that do not appear on the left-hand side (summation), and optionally applies a nonlinearity. One engine therefore runs both neural networks (attention, MLPs) and symbolic reasoning (Datalog over DBF tables) — and can differentiate any program automatically.

Quick start

1. Build the FWH library for your compiler variant (e.g. mfwh_new.bat hb32) so tensorlogic.prg and fwtensor.c are in the lib.
2. From samples\ai, build and run the tutorial companion:

   cd /d C:\fwteam\samples\ai
   C:\fwteam\samples\build_new.bat tltutor hb32
   tltutor.exe

   Output lands in tltutor.log. Walk it side by side with the step-by-step tutorial.
3. Optional speed: put libopenblas.dll on the PATH or next to the exe (or set FWT_BLAS_DLL). Check with ? FWT_BlasActive().

Tensor equations

oTL := TTensorLogic():New()
oTL:Let( "W", { { 1, -1, 0 }, { 0, 1, -1 } } )     // bind tensors (arrays or FW_Tensor)
oTL:Let( "X", { 1, 2, 0.5 } )
oTL:Let( "B", { 2, -2 } )
oTL:Eq( "Y[i] = step( W[i,j] * X[j] + B[i] )" )    // a perceptron, one equation
oTL:Run()                                          // forward chaining to fixpoint
? oTL:GetArray( "Y" )                              // {1, 0}

Indices repeated on the right are joined; indices missing from the left are summed out. Supported nonlinearities: step, sigmoid, relu, tanh, gelu, softmax (last index), lnorm (layer norm, last index, no affine part — express gamma/beta as further equations). Numeric factors scale a term: "S[p,q] = Q[p,k] * K[q,k] * 0.7071".

Equation syntax

Terms with any number of factors are evaluated by chaining pairwise contractions: only the letters still needed for the final output survive each step, so every pairwise step can take the OpenBLAS GEMM fast path.

On parse failure, Eq() returns .F. and sets oTL:cError.

TTensorLogic

DATA: hTensors, aEqs, cError, hPre, hGrads, aParams, hAdamM, hAdamV, nAdamT.

Training loop patterns

// Single-sample / full-batch
oTL:Forward()
oTL:ZeroGrad()
oTL:Seed( "Y", FWT_Scale( FWT_Sub( oTL:Get( "Y" ), tTarget ), 2 ) )  // d MSE
oTL:Backward()
oTL:ClipGrads( 1.0 )
oTL:StepAdam( 0.05 )

// Mini-batch accumulation (TTransformerTL style)
oTL:ZeroGrad()
for i := 1 to nBatch
   oT:AccumStep( aIds[i], aTgt[i], 1 / nBatch )   // param grads add up
next
oTL:ClipGrads( 1.0 )
oTL:StepAdam( nLr )

TTLDomain

Shared entity ↔ index map used by all relations so tensor dimensions match.

TTransformerTL

GPT-style transformer facade over a tensor logic program (~15 equations per layer). Heads are an index (no column slicing): WQ[d,h,k] absorbs the per-head split and WO[h,k,d] absorbs concat+Wo. With identical weights it reproduces the classic Transformer class ForwardSeq() closely, and it trains and generates through the generic autodiff — zero transformer-specific backprop code.

oTLT := TTransformerTL():New( 2, 8, 2, nVocab )
aProbs := oTLT:ForwardSeq( { 2, 3, 4, 5, 6 } )
nLoss  := oTLT:TrainStep( aIds, aTargets, 0.01 )
aOut   := oTLT:Generate( { 1 }, 8 )
// multi-batch:
oTLT:oTL:ZeroGrad()
for i := 1 to nBatch
   oTLT:AccumStep( aWin[i], aTgt[i], 1 / nBatch )
next
oTLT:oTL:ClipGrads( 1.0 )
oTLT:oTL:StepAdam( nLr )

Attention skeleton as equations (from BuildProgram()):

oTL:Eq( "Q[h,p,k] = H0[p,d] * WQ[d,h,k]" )
oTL:Eq( "S[h,p,q] = Q[h,p,k] * K[h,q,k] * 0.5 + OnesH[h] * Mask[p,q]" )
oTL:Eq( "A[h,p,q.] = softmax( S[h,p,q] )" )
oTL:Eq( "O[h,p,k] = A[h,p,q] * V[h,q,k]" )
oTL:Eq( "Att[p,d] = O[h,p,k] * WO[h,k,d]" )
oTL:Eq( "N1[p,d] = lnorm( Att[p,d] + H0[p,d] )" )   // residual + layer norm
// ... FFN, second norm, vocabulary projection, softmax

Datalog over DBF tables

USE tlfam ALIAS FAM NEW              // fields PARENT, CHILD
oDom := TTLDomain():New()
oDom:Harvest( "FAM", { "PARENT", "CHILD" } )       // fix the domain first
oTL:LetDbf( "Parent", "FAM", { "PARENT", "CHILD" }, oDom )
oTL:LetSet( "Male", { "abe", "homer" }, oDom )
oTL:Let( "NotEq", TL_NotEq( oDom:Len() ) )
oTL:Let( "Ancestor", FWT_New( { oDom:Len(), oDom:Len() } ) )

oTL:Eq( "Father[x,y] = step( Parent[x,y] * Male[x] )" )
oTL:Eq( "Sibling[x,y] = step( Parent[p,x] * Parent[p,y] * NotEq[x,y] )" )
oTL:Eq( "Ancestor[x,y] = step( Parent[x,y] + Ancestor[x,z] * Parent[z,y] )" )  // recursive
oTL:Run( 20 )                                      // fixpoint
AEval( oTL:Query( "Ancestor", oDom ), {| a | QOut( a[1] + " -> " + a[2] ) } )

Reasoning in embedding space (the paper's neural-symbolic bridge): embed entities as unit vectors, embed the relation, query it back, and use a temperature sigmoid — T→0 is deductive (hard 0/1), higher T analogical:

oE:Let( "Emb", TL_UnitEmb( n, 256, 1.234 ) )
oE:Eq( "EmbR[i,j] = Parent[x,y] * Emb[x,i] * Emb[y,j]" )
oE:Eq( "Score[x,y] = EmbR[i,j] * Emb[x,i] * Emb[y,j]" )   // ~1 iff (x,y) in Parent
tSoft := FWT_Sigmoid( tShifted, nTemperature )

Helpers (PRG)

C engine (fwtensor.c)

OpenBLAS acceleration

fwtensor.c loads OpenBLAS dynamically at first use — never a hard dependency: libopenblas.dll on the PATH or beside the exe, or the FWT_BLAS_DLL environment variable (set it to off to force the portable C loops; ILP64 builds such as numpy's libscipy_openblas64_ are detected too). FWT_MatMul and every 2-operand FWT_EinSum reducible to a (batched) GEMM run on cblas_sgemm, using BLAS transposes so the common attention patterns need no repacking.

numpy's own OpenBLAS reaches 446 GFLOPS on the 1024 matmul on the same machine — the FWH einsum matches BLAS-class performance; the remaining gap to PyTorch is MKL's AVX-512 kernels and lower per-call overhead, not the language.

Samples

• samples/ai/tltutor.prg — runnable companion of the tutorial (lessons 1–9 → tltutor.log).
• samples/ai/tensorlogic.prg — engine test suite: einsum vs matmul, attention head as 6 equations, Datalog fixpoint, gradient checks vs finite differences, XOR training (9 PASS).
• samples/ai/tltransformer.prg — transformer as equations, parity with the classic class, training + generation (3 PASS).
• samples/ai/tldatalog.prg — Datalog over a family DBF + embedding-space reasoning with temperature (6 PASS).
• samples/ai/tlbench.prg — einsum benchmark, BLAS vs C loops.
• samples/ai/tlminillm.prg — DeepSeek/MiMo-style mini-LLM as pure equations: MLA latent attention, RoPE, mixture-of-experts, multi-token prediction (3 PASS).
• samples/ai/tlfwcode.prg — char-level language model on real FWH sources: Adam + warmup/cosine, batch gradient accumulation, clipping, train/val split, best-val checkpointing and resume, top-k sampling. Configure via TLFW_* env vars; resume with TLFW_RESUME=1.
• samples/ai/tlc.prg — synthetic code-training recipe in miniature: program generator + exact oracle, execution learning (36–40% vs 10% chance) and grammar-constrained decoding (100% valid output even untrained).

Build any sample the same way:

cd /d C:\fwteam\samples\ai
C:\fwteam\samples\build_new.bat <name> hb32
<name>.exe

See Also

• Tensor Logic tutorial (from 0 to 100)
• Transformer Neural Network
• PyTorch-lite roadmap
• Classes Alphabetical