Author here. Collaboration with LLMs has some interesting analogies with the physics af lasers. Medium: LLM, pump energy: conversational prompting, resonant cavity: your judgement and population inversion is what makes the output shine - you are reflecting back ideas that resonate from the training data, each idea builds up on the next.
Agentic frameworks are then designing and engineering the best systems to induce this non-equilibrium state. Hermes quite literally changes the geometry of the cavity itself to induce long-lasting lasing. Is cavity engineering the logical evolution of context engineering?
Good question. In Mnemo, reinforcement and contradiction are handled through slightly different mechanisms. We deliberately don't append — atoms are merged or stored separately but never modified as a log.
When a memory is decomposed into atoms (which includes typing and creating the Beta distribution), and one atom has a similarity threshold >0.9 then it is not stored, but is merged with the existing atom and the Beta distribution is updated: alpha is increased via alpha_new = alpha_orig + alpha_incoming - 1 (subtracting the shared Beta(1,1) prior to avoid double counting).
Contradictory atoms are stored, and the existing atom's Beta distribution is modified by increasing the beta parameter. A graph edge is created with type "contradicts", so in some sense the contradiction history lives in the graph itself. The trade-off with the append-only approach CortexDB takes is that Mnemo can't be used to re-derive past beliefs, as the Beta state is authoritative at any given moment, but its history isn't preserved as a separate log. This raises an interesting question about meta-memory like "Oh I remember I used to think that", but currently this is not a feature in Mnemo.
Mnemo follows the same pattern that retrieval surfaces all relevant atoms, and the model sees the confidence and can come to a belief. The chain of updates is not stored, the Beta distribution is updated in-place.
The typing is done by a small model (Claude Haiku currently), and once typed, we don't revisit the reclassification. To be honest I haven't revisited that design choice, and I'm curious to understand the cases where an atom should be reclassified. We have made the time decay different for each type, following cognitive research reported by McClelland, McNaughton and O'Reilly in 1995, so reclassifying would be difficult (but not impossible).
Mnemo, SOTA on LoCoMo multi-hop, is an agentic memory system that was designed for agents, with agentic experience (AX) in mind. Its differentiator is the ability to share memories with other agents, addressing the institutional knowledge transfer problem that multi-agent systems are currently running into and simulataneously allowing for the transfer of skills amongst agents within an organization.
The Mnemo system is based on typed atoms -- episodic, procedural and semantic -- reflecting what has been discussed in the field of cognitive science since Endel Tulving work in the 1970s. Each memory is broken into a set of these typed
atoms and assigned a confidence score using a Beta distribution, which are then updated in a Bayesian process as further memories either confirm or contradict the stored atoms. This implements what E. T. Jaynes wrote about in his famous textbook "Probability: The Logic of Science" where he described how one would hypotheticlly teach a robot to use the scientific method. The atoms are stored in a knowledge graph with edges that span atoms from multiple memories -- even shared atoms from other agents.
Memory systems are not document stores, as memories are imprecise and sometimes contradictory. Mnemo embraces this, and surfaces the contradictions for the cognitive processor to deal with -- in this case AI agents based on LLMs.
Mnemo also has a dreaming process where the confidence of the atoms are degraded
over time (with a different half life for each type), similar atoms are consolidated and graph edges are connected which may have been overlooked at the time of "remembering".
This structure has resulted in Mnemo achieving SOTA in the multi-hop category of the LoCoMo benchmark, arguably the most difficult and most important category for a memory system.
I think this is a very important question, and it makes it clear that memory systems are less about fact retrieval, and more about knowledge classification. Memories systems are not document stores -- which to be fair this hippo system does recognize and motivates by exponential decay, recall strengthening and "sleep" consolidation.
I personally don't think a memory system should try to "select what to forget", but to store everythign and live with the contradictions inherent in history. Having said that, we need to ascribe a certain confidence to each memory at storage time, where something uncertain is described as such, and when contradicting information gets stored, it reduces the confidence even further -- this on top of time decay and retreival bumps in confidence. E. T. Jaynes argued that this could be achieved in machines through Bayesian updating, say a beta distribution is stored for each memory and upon storing knowledge that confirms this memory, the beta distribution is updated to have more confidence (the original is the prior).
If every memory has a Bayesian prior denoting confidence, and this is surfaced when recalling, then the LLM itself can decide how to sythesize the different memories. Together with a "remembered on" field, the LLM can grok that the database schema was changed, or a certain design pattern was discarded (for example).
(Full disclosure, I have developed a memory system myself which I will post here in a couple days, with a slightly different target audience than hippo).
Agentic frameworks are then designing and engineering the best systems to induce this non-equilibrium state. Hermes quite literally changes the geometry of the cavity itself to induce long-lasting lasing. Is cavity engineering the logical evolution of context engineering?
Happy to hear your thoughts.