The big benefit in my experience was that I could have a program with real users, that did have errors (from me being new to Elixir and not knowing better) and still not experiencing downtime.
Instead, CPU or Memory would increase over time, hit the VM limit, kill and restart.
So later when I noticed this, I could debug and fix it without simultaneously fighting a prod incident.
I wish the nix programming language wasn't so rough because it can be pretty great at this problem. Being able to compile from source while just listing out package dependencies is powerful.
Yep, prototype exactly that this past week. With a strong instruction spec prompt from the start, you can have an AI come up with a much better answer by making sure it knows it has time to answer the questions and how it should approach the problem in stages.
The great part is with clear enough directions it also knows how to evaluate whether its done or not.
It seems plausible you could have the LLM side call upon its knowledge of known problems and answers to quiz the q-learning side.
While this would still rely on a knowledge base in the LLM, I would imagine it could simplify the effort required to train reinforcement learning models, while widening the domains it could apply to.
Instead, CPU or Memory would increase over time, hit the VM limit, kill and restart.
So later when I noticed this, I could debug and fix it without simultaneously fighting a prod incident.