FiveTech Support Forums

FiveWin / Harbour / xBase community
Board index latest AI news LLMs from Scratch
Posts: 44162
Joined: Thu Oct 06, 2005 05:47 PM
LLMs from Scratch
Posted: Sat Sep 27, 2025 09:31 AM
FWH provides its own Class Transformer though we are still refining it.

In the meantime we save all the related tech info we find:
https://youtu.be/p3sij8QzONQ?si=OWkORRqjF-bEx8U9
https://github.com/rasbt/LLMs-from-scratch

We do appreciate whatever info you may find about it :wink:
regards, saludos

Antonio Linares
www.fivetechsoft.com
Posts: 44162
Joined: Thu Oct 06, 2005 05:47 PM
Re: LLMs from Scratch
Posted: Sat Sep 27, 2025 04:02 PM
</s>graph TD
    subgraph "1. Preparaci贸n de Entrada"
        A[IDs de Palabras] --&gt; B(Tabla de Embeddings)
        C[Posiciones] --&gt; D(Tabla Posicional)
        B --&gt; E{Suma}
        D --&gt; E
        E --&gt; F[Input 'x']
    end

    F --&gt; G(Bucle a trav茅s de N Capas)

    subgraph "Interior de UNA Capa Transformer"
        G_in(Entrada a la capa) --&gt; H[Atenci贸n Multi-cabeza]
        H --&gt; I{Suma}
        G_in --&gt;|Conexi贸n Residual| I
        I --&gt; J[Normalizaci贸n]
        J --&gt; K[Feed Forward]
        K --&gt; L{Suma}
        J --&gt;|Conexi贸n Residual| L
        L --&gt; M[Normalizaci贸n]
        M --&gt; G_out(Salida de la capa)
    end
    
    G --&gt; G_in
    G_out --&gt; G

    subgraph "3. Generaci贸n de Salida Final"
        G -- Final de N capas --&gt; N(Promedio)
        N --&gt; O(Capa Lineal/Multiplicaci贸n)
        O --&gt; P[Logits]
        P --&gt; Q(Funci贸n Softmax)
        Q --&gt; R[Probabilidades]
    end<e>
regards, saludos

Antonio Linares
www.fivetechsoft.com

Continue the discussion