Nice - good to see activity in this space, the more folks are on it the quicker we'll get to a world with a satisfactory solution.
I like your approach - I went down a bit more of a swiss-army-knife one with a hybrid.
"Session memory" where turns are stored in full so that the full context can be retrieved if an important fact is missed in compaction. After a certain time sessions are compacted. I'm currently evaluating a model where only a certain number of turns per session are kept plain-text and compaction happens as a sliding window
"knowledge memory" All info is periodically fed into a knowledge graph extraction and a knowledge graph is built and indexed
"memory chunking" chunks of memory are stored individually into a vector space where they can be retrieved through similarity search as well as standard semantic searches.
My theory is that giving the agent flexibiity to query the tool best suited for it's current needs is the way to go, as agents/llms become better, they'll only get better at summarisation and tool choice.
Couple reasons I like storing more data over less is that storage is cheap and if compaction misses details, you can prompt your agent to go back to that session you had about X last week and extract that detail.
Similar to you however, it's still also very early stages and the ground is shifting rapidly. Claude Codes HTTP hooks are a great step towards a model that I believe can work assuming it gets adopted as a standard. A big problem of having memory be portable is that you gotta be able to plug it into any client you want, and for the time being, MCP + system prompt is the only option - and even that one's more than flaky.
Would be awesome to chat to someone from OpenAI or Anthropic on their take and pow-wow on options
At the moment I'm doing a hybrid but that's still something that needs a lot of trial and error to get right. Doing turn-based context with auto compacting at the moment, but I'm sure that'll change a few more times before it works well
Update from two days later: we shipped the Claude Code HTTP hooks integration.
The injection problem I mentioned earlier, of memory not loading automatically without system prompt config is now partially solved for Claude Code. SessionStart hook auto-injects memory capabilities, TaskCompleted tracks turns, SessionEnd persists everything.
It's also clarified our thinking on the protocol question: lifecycle hooks (SessionStart, SessionEnd, ContextCompaction) feel like the right abstraction for ambient memory. If other tools did the same, you'd configure memory once and it would work everywhere.
That is what I do at the moment - I gotta update the instructions on the website to prod users to set things up this way.
The issue I perceive though is that adding an MCP server alone is not enough to modify the system prompt of the AI Agent. I tried to have the mcp server description be an injected prompt to add these instructions to the system prompt but that doesn't seem to work, I tried adding sampling to the MCP server which supposedly should be able to plug into messages without luck, tried to optimise for chatGPT with an OpenAPI spec etc.
The only way I found that I can get those clients to use my memory layer is by doing what you describe - which is not necessarily the most user friendly/one-click setup I desire
I'm not gonna lie - at the moment it's pretty basic as a chunked semantic store where relevant chunks are retrieved in conjunction with some criteria the Agent can pass to the MCP server.
Context usage is definitely a problem that's on my mind also and semantic anchors are one area I'm exploring but don't have a clear architecture for it jotted down yet. The real problem I'm facing right now is how to inject this into say claude or chatgpt and have those agents default use it as a memory layer
Short answer is yes - just add the MCP server and you're golden. Longer answer is that most chat clients do not allow the MCP server to automatically inject it's own system prompt which means you have to specifically prompt your AI to write to memobase.
Right - that makes sense. I see it similarly. My assumption though is that as models get better, the likelyhood of the model missing context that matters the most will get lower and lower.
The hybrid approach though is nice, I'll have a think about that and see if that's something I can incoporate into it. Thanks for the feedback, very much appreciated
Yeah, 100% agree. That's one thing I just thought about yesterday also - Every session should be summarised and written to the data store also, that way sessions become portable contexts. There's potentially a case to be made to have full replay, i.e. the plaintext sessions stored but I'm not entirely sure on how much more valuable that is over a summary.
I like your approach - I went down a bit more of a swiss-army-knife one with a hybrid.
"Session memory" where turns are stored in full so that the full context can be retrieved if an important fact is missed in compaction. After a certain time sessions are compacted. I'm currently evaluating a model where only a certain number of turns per session are kept plain-text and compaction happens as a sliding window "knowledge memory" All info is periodically fed into a knowledge graph extraction and a knowledge graph is built and indexed "memory chunking" chunks of memory are stored individually into a vector space where they can be retrieved through similarity search as well as standard semantic searches.
My theory is that giving the agent flexibiity to query the tool best suited for it's current needs is the way to go, as agents/llms become better, they'll only get better at summarisation and tool choice.
Couple reasons I like storing more data over less is that storage is cheap and if compaction misses details, you can prompt your agent to go back to that session you had about X last week and extract that detail.
Similar to you however, it's still also very early stages and the ground is shifting rapidly. Claude Codes HTTP hooks are a great step towards a model that I believe can work assuming it gets adopted as a standard. A big problem of having memory be portable is that you gotta be able to plug it into any client you want, and for the time being, MCP + system prompt is the only option - and even that one's more than flaky.
Would be awesome to chat to someone from OpenAI or Anthropic on their take and pow-wow on options