hycpax·昨年·議論> When a LLM outputs a word, it commits to that word, without knowing what the next word is going to be.Please, people, read before you write. Both the article and the paper explain that that’s not how it works.‘One token at a time’ is how a model generates its output, not how it comes up with that output.> That is kind of weird. Why would you do that, and how would you be sure?The model is sure because it doesn’t just predict the next token. Again, the paper explains it.
hycpax·昨年·議論Only starts approaching PPL of AR at block size 4. May as well just use multi-token prediction with a standard AR model.
Please, people, read before you write. Both the article and the paper explain that that’s not how it works.
‘One token at a time’ is how a model generates its output, not how it comes up with that output.
> That is kind of weird. Why would you do that, and how would you be sure?
The model is sure because it doesn’t just predict the next token. Again, the paper explains it.