Yeah that’s what they try to do with the latest coding agents sub agents which only have the context they need etc. but atm it’s too much work to manage contexts at that level
A developer can blast millions of tokens in minutes. When you have a context size of 250k that’s just 4 queries. But with tool usage and subsequent calls etc it can easily just do many millions in one request
But if you just ask a question or something it’ll take a while to spend a million tokens…
Once communication with my customers moved to teams. I've had a very hard time to find historical agreements and decisions.
I try very hard to create a robust system for ADR logging now. And not just for system architecture. But for all decisions and agreements in my projects and across changes.
I often find myself clearing the context when dealing with llms to get a fresh take. Often it just has so much context reinforcing its previous decisions.
Not sure if the author tried to just start a new thread. But anyway, for now you always need to keep an eye on these things and manage it if it follows red herrings or ends up in some logical loop
Sidenote : newlines is one thing tat can be quite tricky for llms in general.
If the company has enough grc red tape, integrating with slack can become almost impossible I can imagine