Mnemo: Shareable typed agentic memory system with Bayesian belief updating(github.com)
github.com
Mnemo: Shareable typed agentic memory system with Bayesian belief updating
https://github.com/inforge-ai/mnemo-server
4 comments
Disclosure: I've been exploring CortexDB (cortexdb.ai) for a project, so the write-path question is top of mind. Bayesian belief updating is an interesting angle for confidence-weighted retrieval, but the question I'd push on is: when a belief updates, does the underlying memory get rewritten, or does the system append a new observation and recompute the posterior? CortexDB takes the latter approach — append-only events, beliefs derived at query time — which preserves the raw signal so you can re-derive beliefs whenever your model improves. Typed atoms (episodic/procedural/semantic) face the same issue: once you've committed to a classification at ingest, walking it back is hard. How does Mnemo handle re-derivation when the belief model improves?
Good question. In Mnemo, reinforcement and contradiction are handled through slightly different mechanisms. We deliberately don't append — atoms are merged or stored separately but never modified as a log.
When a memory is decomposed into atoms (which includes typing and creating the Beta distribution), and one atom has a similarity threshold >0.9 then it is not stored, but is merged with the existing atom and the Beta distribution is updated: alpha is increased via alpha_new = alpha_orig + alpha_incoming - 1 (subtracting the shared Beta(1,1) prior to avoid double counting).
Contradictory atoms are stored, and the existing atom's Beta distribution is modified by increasing the beta parameter. A graph edge is created with type "contradicts", so in some sense the contradiction history lives in the graph itself. The trade-off with the append-only approach CortexDB takes is that Mnemo can't be used to re-derive past beliefs, as the Beta state is authoritative at any given moment, but its history isn't preserved as a separate log. This raises an interesting question about meta-memory like "Oh I remember I used to think that", but currently this is not a feature in Mnemo.
Mnemo follows the same pattern that retrieval surfaces all relevant atoms, and the model sees the confidence and can come to a belief. The chain of updates is not stored, the Beta distribution is updated in-place.
The typing is done by a small model (Claude Haiku currently), and once typed, we don't revisit the reclassification. To be honest I haven't revisited that design choice, and I'm curious to understand the cases where an atom should be reclassified. We have made the time decay different for each type, following cognitive research reported by McClelland, McNaughton and O'Reilly in 1995, so reclassifying would be difficult (but not impossible).
When a memory is decomposed into atoms (which includes typing and creating the Beta distribution), and one atom has a similarity threshold >0.9 then it is not stored, but is merged with the existing atom and the Beta distribution is updated: alpha is increased via alpha_new = alpha_orig + alpha_incoming - 1 (subtracting the shared Beta(1,1) prior to avoid double counting).
Contradictory atoms are stored, and the existing atom's Beta distribution is modified by increasing the beta parameter. A graph edge is created with type "contradicts", so in some sense the contradiction history lives in the graph itself. The trade-off with the append-only approach CortexDB takes is that Mnemo can't be used to re-derive past beliefs, as the Beta state is authoritative at any given moment, but its history isn't preserved as a separate log. This raises an interesting question about meta-memory like "Oh I remember I used to think that", but currently this is not a feature in Mnemo.
Mnemo follows the same pattern that retrieval surfaces all relevant atoms, and the model sees the confidence and can come to a belief. The chain of updates is not stored, the Beta distribution is updated in-place.
The typing is done by a small model (Claude Haiku currently), and once typed, we don't revisit the reclassification. To be honest I haven't revisited that design choice, and I'm curious to understand the cases where an atom should be reclassified. We have made the time decay different for each type, following cognitive research reported by McClelland, McNaughton and O'Reilly in 1995, so reclassifying would be difficult (but not impossible).
Mnemo, SOTA on LoCoMo multi-hop, is an agentic memory system that was designed for agents, with agentic experience (AX) in mind. Its differentiator is the ability to share memories with other agents, addressing the institutional knowledge transfer problem that multi-agent systems are currently running into and simulataneously allowing for the transfer of skills amongst agents within an organization.
The Mnemo system is based on typed atoms -- episodic, procedural and semantic -- reflecting what has been discussed in the field of cognitive science since Endel Tulving work in the 1970s. Each memory is broken into a set of these typed atoms and assigned a confidence score using a Beta distribution, which are then updated in a Bayesian process as further memories either confirm or contradict the stored atoms. This implements what E. T. Jaynes wrote about in his famous textbook "Probability: The Logic of Science" where he described how one would hypotheticlly teach a robot to use the scientific method. The atoms are stored in a knowledge graph with edges that span atoms from multiple memories -- even shared atoms from other agents.
Memory systems are not document stores, as memories are imprecise and sometimes contradictory. Mnemo embraces this, and surfaces the contradictions for the cognitive processor to deal with -- in this case AI agents based on LLMs.
Mnemo also has a dreaming process where the confidence of the atoms are degraded over time (with a different half life for each type), similar atoms are consolidated and graph edges are connected which may have been overlooked at the time of "remembering".
This structure has resulted in Mnemo achieving SOTA in the multi-hop category of the LoCoMo benchmark, arguably the most difficult and most important category for a memory system.
Mnemo is currenlty in design phase, if anyone wishes to beta test please reach out. Full paper with methodology breakdown: https://github.com/inforge-ai/mnemo-server/blob/main/paper/m... https://solitonmaths.substack.com/p/the-persistence-of-ai-me...
The Mnemo system is based on typed atoms -- episodic, procedural and semantic -- reflecting what has been discussed in the field of cognitive science since Endel Tulving work in the 1970s. Each memory is broken into a set of these typed atoms and assigned a confidence score using a Beta distribution, which are then updated in a Bayesian process as further memories either confirm or contradict the stored atoms. This implements what E. T. Jaynes wrote about in his famous textbook "Probability: The Logic of Science" where he described how one would hypotheticlly teach a robot to use the scientific method. The atoms are stored in a knowledge graph with edges that span atoms from multiple memories -- even shared atoms from other agents.
Memory systems are not document stores, as memories are imprecise and sometimes contradictory. Mnemo embraces this, and surfaces the contradictions for the cognitive processor to deal with -- in this case AI agents based on LLMs.
Mnemo also has a dreaming process where the confidence of the atoms are degraded over time (with a different half life for each type), similar atoms are consolidated and graph edges are connected which may have been overlooked at the time of "remembering".
This structure has resulted in Mnemo achieving SOTA in the multi-hop category of the LoCoMo benchmark, arguably the most difficult and most important category for a memory system.
Mnemo is currenlty in design phase, if anyone wishes to beta test please reach out. Full paper with methodology breakdown: https://github.com/inforge-ai/mnemo-server/blob/main/paper/m... https://solitonmaths.substack.com/p/the-persistence-of-ai-me...
[deleted]