An analysis and chain of thoughts Continual learning,memory and context problem(siliconandsoul.substack.com)
siliconandsoul.substack.com
An analysis and chain of thoughts Continual learning,memory and context problem
https://siliconandsoul.substack.com/p/continual-learningmemory-and-context
2 comments
Tried connecting the dots between continual learning, memory, and context limits in LLMs, and how this lines up with ideas from the Nested Learning paper. The core gap seems to be the same: models can process more tokens, but they still don’t accumulate knowledge over time. Long context and RAG look like scaffolding; nested or hierarchical learning feels closer to what persistent, evolving intelligence would actually require.