FiveTech Support Forums

Mixture-of-Recursions (MoR) is a recursive Transformer. But instead of looping the same number of times for all tokens, MoR assigns each token a different recursion depth depending on how much “thinking” it needs. The router, a tiny neural net, decides this, token by token, during training and inference.