Dont be fooled. Reasoning does not happen in the prediction of the next token, it happens virtually in the text that is created.
The next token prediction is just "the hardware" following the underlying rules. Like the basic set of rules.. in a sense similar to how the "game of life" does not really contain gliders. Gliders are just a self stabilised system that arrises from the simple rules.
This is trivially false, the text gets transformed into activations for the weights. If there is reasoning it's in the connection pattern of the weights.
The fact that the output produces one token at a time does not mean that the LLM's internal state is processing just the next token
reliablereason · · focus · HN ↗
The next token prediction is just "the hardware" following the underlying rules. Like the basic set of rules.. in a sense similar to how the "game of life" does not really contain gliders. Gliders are just a self stabilised system that arrises from the simple rules.
WhyIsItAlwaysHN · · focus · HN ↗
The fact that the output produces one token at a time does not mean that the LLM's internal state is processing just the next token