What happens between entering the prompt and seeing the first word appear(shbhmrzd.github.io)

shbhmrzd.github.io

What happens between entering the prompt and seeing the first word appear

https://shbhmrzd.github.io/ai/ml-foundations/llm-training/2026/07/11/using-the-trained-model.html

2 comments

shbhmrzd19시간 전

I have been writing a series on understanding LLMs from the ground up for software engineers. This is the last part.

In this post I cover what happens when you type a prompt and hit enter. How the model generates one token at a time, why that is slow, what the KV cache does about it,and how decoding strategies like temperature, top-k, and top-p shape the response.

Purposefather19시간 전