It is interesting to compare this to LLMs - they also have the bounded context that you can see as the analogue to our working memory. It can contain enormously more bits of information than the 4 things the article says is the capacity of our working memory - but the 4 things can probably be much more complex internally - they are more like 4 pointers probably.
But at some level context engineering is very similar to what this article talks about.
The feature I am waiting for in all of these editors is integrating 'red lining' as a channel for LLM input. This is the best interface for working on a text. https://www.roughdraft.md/ does the core idea pretty well - but is not well integrated with the rest (browsing, etc).
If the action is decided by code based on metadata - then what is really the LLM task? And if you say that it is only the type of action that is decided by code - then this is maybe a mitigation - but the llm still can do a lot of harm. And also it is very limiting - using the llm to decide the action is very useful. This is different from SQL injection - where the action is determined by the code and the injection is really making a code parsing error.
It might still be the way to go - but calling it 'the real solution' is overselling it.
They do predict what injections might be effective - so it is a theory. I don't know how novel it is and it is not very deep (as you noted the general mechanism is quite obvious) - but they do it quite systematically so it is useful.
I am open to changing my mind and I am looking forward for answers to your question - but I think it is like a click bait - looks interesting - but not really useful.
I am building my self-hosting llm-wiki system (https://gist.github.com/karpathy/442a6bf555914893e9891c11519...). My approach is to start with a theory of how such systems could work. Then since llms can interpret theory - this theory becomes an executable llm-wiki system itself.
"In the 890s, having recently converted to Orthodox Christianity, Boris ensured his church would be independent from the Patriarchate of Constantinople." --- I thought Orthodox Christianity was created by the Great Schism in 1054.
First you need to write these specifications and if you say just tell the llm to write them - then how would it be different from just tell the llm to write the program?
I guess you can argue that these are two independent processes so you can combine them to get something more reliable than both - this might be a viable path. But from what I heard writing formal specifications is just really hard - I haven't seen anything practical in this area.
I concur - it does not make sense to do in llm prompts what can be done in code. Code is cheaper, faster, deterministic and we have lots of experience with working with code.
I was also confused by the tag menu - I thought they are sections in the article - I completely failed here :(
By the way I have a page with some more sources for the context degradation phenomenon: https://zby.github.io/commonplace/notes/agent-context-is-con...