We need to talk about LLM's and non-determinism(rdrocket.com)

3 pointsby decodebytes10 miesięcy temu1 comments

We need to talk about LLM's and non-determinism

https://www.rdrocket.com/blog/we-need-to-talk-about-LLMs-non-determinism

1 comments

reneberlin10 miesięcy temu

I also see many people complaining about non-determinism, mainly from products like ChatGPT, that they have no control over many parts (system-prompt, llm, ...) and the most interesting in this case: SEED-value.

If you are on the same hardware and run your model with the same seed-value, and the same prompt: the output should be deterministic for the most architectures and models we talk about.

What i meant is, that most people don't even know if they are using a seed-value or not in their chatbot. And having no seed-value set, or having the seed-value set to another value on purpose by the provider makes the whole reception understandable.