AI / ML Glossary
Definitions for artificial intelligence and machine learning terminology relevant to FiveWin's AI classes, neural network infrastructure, and LLM integration. Terms cover model architectures, training concepts, optimization algorithms, and the supported AI providers. Organized alphabetically.
Adam / SGD
Two optimizers provided by FWH. Adam (Adaptive Moment Estimation) maintains per-parameter adaptive learning rates with bias-corrected momentum, suitable for complex models. SGD (Stochastic Gradient Descent) with optional momentum is simpler and preferred for less complex training scenarios.
API Key
An authentication credential required by cloud AI providers. FWH's AI classes read keys from environment variables (OPENAI_API_KEY, DEEPSEEK_API_KEY, GEMINI_API_KEY) or accept them directly in the New() constructor. The local TOLlama class requires no API key.
Attention
A mechanism in Transformer networks that computes weighted relevance scores between every pair of positions in a sequence. FWH's MultiHeadAttention class implements scaled dot-product attention with multiple parallel heads, enabling the model to focus on different representational subspaces.
Backpropagation
The core training algorithm for neural networks, computing gradients of the loss function with respect to each weight via the chain rule. FWH's Transformer and neural network classes include Backward() methods that propagate error gradients backward through the computation graph.
ChatGPT / OpenAI
OpenAI's GPT model family accessed via REST API. FWH's TOpenAI class integrates with models including gpt-4o and gpt-4o-mini, supporting text prompts, vision (image analysis via SendImage()), and streaming responses through callbacks.
Cosine Distance
A similarity metric measuring the cosine of the angle between two vectors, ranging from 1 (identical direction) to 0 (orthogonal) to -1 (opposite). Used in FWH's TEmbeddings class to compare embedding vectors for semantic similarity search and clustering.
DeepSeek
An AI model family by DeepSeek known for strong coding and reasoning capabilities. FWH's TDeepSeek class connects to DeepSeek's API, supporting models deepseek-chat for general conversation and deepseek-reasoner for chain-of-thought reasoning.
Embedding
A dense numerical vector representation of text, capturing semantic meaning in a high-dimensional space. In FWH, embeddings are generated by the TEmbeddings class and used for similarity search, clustering, and RAG (Retrieval-Augmented Generation) pipelines.
Epoch
One complete pass through the entire training dataset during neural network training. FWH's training loops iterate over configurable numbers of epochs, with each epoch comprising forward pass, loss computation, backpropagation, and parameter updates.
Fine-tuning
The process of further training a pre-trained model on domain-specific data to adapt its capabilities. FWH supports fine-tuning workflows through its neural network and Transformer training infrastructure, where pre-trained weights are adjusted on custom datasets.
Gemini
Google's multimodal AI model family. FWH's TGemini class integrates with the Google Generative Language API, supporting text prompts, image file uploads for visual analysis, and optional streaming callbacks for real-time response.
Grok / xAI
Elon Musk's xAI model family. FWH's TGrok class provides integration with the Grok API for text generation and chat interactions, following the same pattern as other FWH AI provider classes.
Inference
The process of using a trained model to make predictions on new, unseen data (as opposed to training). FWH's AI classes perform inference when calling Send() with a prompt -- the model processes the input and generates a response without modifying its weights.
Kimi / Moonshot
Moonshot AI's Kimi model family, known for long-context capabilities (up to 2 million tokens). FWH's TKimi class provides integration with the Moonshot API for text generation, following the standard FWH AI class pattern.
LLM (Large Language Model)
A neural network with hundreds of millions to trillions of parameters, trained on vast text corpora to generate human-like text. FWH integrates with multiple LLMs through dedicated classes: TOpenAI, TDeepSeek, TGemini, TGrok, TKimi, and TOLlama for local models.
Loss Function
A function measuring the discrepancy between predicted and actual values during training. FWH's training infrastructure supports cross-entropy loss for classification tasks (comparing predicted probability distributions to one-hot targets) and MSE (Mean Squared Error) for regression.
Neural Network
A computing system inspired by biological neural networks, consisting of layers of interconnected nodes (neurons) with learned weights. FWH provides the TNeuralNetwork class supporting feed-forward, convolutional, and recurrent architectures for custom ML tasks.
Ollama
A local LLM server that runs models on the user's own machine without cloud dependencies or API keys. FWH's TOLlama class connects to Ollama's REST API at http://localhost:11434/api/chat, supporting all models served by Ollama (llama3, deepseek-r1, mistral, etc.).
Optimizer
An algorithm that updates neural network weights to minimize the loss function during training. FWH provides optimizer implementations in transformer.prg with configurable learning rates, weight decay, and momentum parameters.
RAG (Retrieval-Augmented Generation)
A technique combining information retrieval with LLM generation: relevant context chunks are retrieved from a knowledge base (via embedding similarity search) and injected into the prompt. FWH's TEmbeddings class and matrix operations enable building complete RAG pipelines.
ReLU
Rectified Linear Unit activation function: ReLU(x) = max(0, x). The default activation in FWH's FeedForward transformer layers and TNeuralNetwork hidden layers, valued for its computational efficiency and mitigation of the vanishing gradient problem.
Sigmoid
An activation function: sigmoid(x) = 1 / (1 + exp(-x)), squashing real-valued inputs to the (0, 1) range. Used in FWH's neural network output layers for binary classification and as a gating mechanism in various network architectures.
Softmax
An activation function that converts a vector of raw scores (logits) into a probability distribution over classes, where all outputs sum to 1. Used in FWH's Transformer output layer for next-token prediction and in neural network classifiers.
Temperature
A sampling parameter (typical range 0.0 to 2.0) controlling the randomness of LLM output. Lower values (0.0-0.3) produce more focused, deterministic responses; higher values increase creativity and diversity. FWH's AI classes expose nTemperature as a configurable DATA member.
Token
The atomic unit of text that an LLM processes. Tokens can be words, subwords (e.g., "un" + "believe" + "able"), or characters. FWH's AI classes interact with APIs that bill by token count and limit context windows by maximum tokens. The Transformer class includes its own token embedding lookup table.
Transformer
The neural network architecture underlying modern LLMs, introduced in "Attention Is All You Need" (Vaswani et al., 2017). FWH provides a pure Harbour/C implementation of the Transformer encoder in source/classes/transformer.prg with multi-head self-attention, feed-forward layers, layer normalization, and positional encoding.
Vector
An array of floating-point numbers representing data in a mathematical space. In AI contexts within FWH, vectors represent embeddings (semantic meaning), neural network weights, and intermediate activations. Fast vector operations are implemented in the C extension matrixes.c.