FiveTech Support Forums
FiveWin / Harbour / xBase community
Board index Artificial Intelligence examples Transformers diagram
</s>flowchart TD A[Tech Tree] --> B[Python] %% --- Dataset --- subgraph DS[Dataset] C[Basic Dataset] --> R[Data Collection] R --> S[Data Cleaning] S --> T[Tokenisation] T --> U[Byte Pair Encoding] U --> V[Causal Language Modelling] V --> W[Positional Encoding] W --> X[Rotary Positional Encoding] end %% --- Operaciones Básicas --- subgraph OP[Operaciones Básicas] D[Tensor] E[Basic Operations] Y[Linear Layer] Z[ReLU] AA[Swish] AB[GeLU] AC[SiLU/Swish] AD[Attention] AE[Multi-Head Attention] end %% --- Entrenamiento --- subgraph TR[Entrenamiento] F[Autograd] G[Mean Squared Error] H[Cross Entropy] I[Backpropagation] J[Stochastic Gradient Descent] K[Weight Decay] L[Momentum] M[Learning rate] N[Gradient Clipping] O[AdamW] AN[Hyperparameter Tuning] AO[μ-Scaling Laws] end %% --- Arquitectura --- subgraph ARQ[Arquitectura] AF[Multi-Layer Perceptron] Q[Transformer Block] AG[Transformer] AH[Small Language Model] AI[Modern Language Model] AJ[Quantisation] AK[Batch Norm] AL[Layer Norm] AM[RMSNorm] end %% Relaciones principales B --> C B --> D B --> E D --> F F --> G F --> H F --> I I --> J J --> K J --> L J --> M J --> N J --> O O --> Q E --> Y Y --> Z Y --> AA Y --> AB AA --> AC Y --> AD AD --> AE Z --> AF AA --> AF AB --> AF AC --> AF AF --> Q AE --> Q I --> Y Q --> AG AG --> AH AH --> AI AI --> AJ AH --> AK AH --> AL AH --> AM K --> Q L --> Q M --> Q N --> Q AN --> AO --> Q <e>