What happens between entering the prompt and seeing the first word appear(shbhmrzd.github.io)
shbhmrzd.github.io
What happens between entering the prompt and seeing the first word appear
https://shbhmrzd.github.io/ai/ml-foundations/llm-training/2026/07/11/using-the-trained-model.html
In this post I cover what happens when you type a prompt and hit enter. How the model generates one token at a time, why that is slow, what the KV cache does about it,and how decoding strategies like temperature, top-k, and top-p shape the response.