EverMemOS: When Memory Stops Being a Junk Drawer

Memory sounds simple until the assistant has to remember two incompatible things at once.

A customer loves craft beer. The same customer is temporarily taking antibiotics. A flat memory system retrieves “likes IPA” and recommends a variety pack, because apparently “memory” means grabbing the loudest sticky note from a drawer and pretending it is wisdom. A more useful assistant retrieves the preference, the medical constraint, the timing, and the relation among them. It recommends a mocktail and quietly avoids turning personalization into negligence.

That is the practical point behind EverMemOS, a paper proposing a self-organizing memory operating system for long-horizon LLM agents.¹ The paper is not mainly about giving agents more storage. It is about changing what storage becomes before the model reasons over it.

The usual mental shortcut says: long-term AI memory equals bigger context windows, better vector databases, or more retrieval. EverMemOS makes a sharper claim. The failure mode is often not that the system forgot the right fact. The failure is that it remembered several fragments without consolidating them into a coherent, time-aware user state.

That difference matters. A support bot, sales copilot, learning tutor, healthcare-adjacent coach, or internal enterprise assistant does not merely need “relevant snippets.” It needs a way to turn messy interaction history into usable context: stable enough to support continuity, flexible enough to update when the user changes, and selective enough not to flood the model with every vaguely related memory.

EverMemOS attacks that problem through a three-stage lifecycle:

Dialogue stream
    ↓
MemCells: episodic traces + atomic facts + time-bounded foresight
    ↓
MemScenes: consolidated thematic memory structures + user profile updates
    ↓
Reconstructive recollection: query-time context assembled only as needed

The important word is lifecycle. Memory is not treated as an archive. It is treated as something that forms, consolidates, and gets reconstructed for use. Annoyingly biological, yes. Operationally useful, also yes.

The real enemy is fragmented correctness

Many enterprise memory systems fail in a boring way. They retrieve true things and still answer badly.

That sounds paradoxical only if we assume retrieval is the same as reasoning. It is not. Retrieval answers the question: “Which pieces of past text look relevant?” Long-horizon reasoning asks something harder: “Which past events jointly define the current state of the user, task, constraint, or decision?”

EverMemOS is designed around this gap. The paper argues that existing memory systems often store isolated records and retrieve fragments. This helps with local recall, but it does not automatically solve conflict detection, temporal validity, or profile stability. A user’s preference from last month and a temporary constraint from last week are both relevant; the answer depends on their relationship.

That is why a mechanism-first reading is better than a leaderboard-first reading. If we start with the benchmark table, EverMemOS looks like another system claiming a few more points over Zep, MemOS, Mem0, MemoryOS, and MemU. Fine. Leaderboards must eat too. But the interesting business lesson is not “System X wins.” It is why structured memory changes the type of context the model receives.

The paper’s architecture has three moving parts. Each fixes a different version of the junk-drawer problem.

This is the key correction: the paper is not saying “retrieve more.” It is saying “retrieve from memory that has already been organized.”

Phase I turns conversation sludge into MemCells

EverMemOS begins with Episodic Trace Formation. The system takes continuous interaction history and segments it into discrete memory units called MemCells.

A MemCell is not just a stored message. Formally, it contains four components:

• an Episode, a concise third-person narrative of the event;
• Atomic Facts, discrete verifiable statements extracted from the episode;
• Foresight, forward-looking inferences or temporary states with validity intervals;
• Metadata, including timestamps and source pointers.

This design is less glamorous than inventing a new model and much closer to the unromantic work that makes enterprise systems usable. Raw dialogue is repetitive, full of pronouns, partial references, corrections, and “as I said earlier” debris. If a memory system stores it directly, later retrieval has to solve both search and interpretation at the same time.

EverMemOS separates the two. First, it rewrites a segment into a stable episode. Then it extracts facts for precise matching. Then it adds time-bounded foresight, such as a plan, temporary condition, upcoming trip, or short-term constraint.

That last part is easy to underestimate. Most business memory is not timeless. A customer’s delivery address, product interest, payment issue, hiring constraint, or compliance status may be valid only for a period. Treating all memory as eternal is how “personalization” becomes a very polite form of being wrong.

A useful way to read Phase I is this:

Raw interaction:
    "I’m going to Beijing next week. Any suggestions?"

MemCell:
    Episode: user discussed an upcoming Beijing trip
    Atomic facts: user will travel to Beijing next week
    Foresight: prepare documents, clothing, itinerary, reservations
    Metadata: timestamp, source dialogue

The system is not simply remembering the sentence. It is preparing the sentence for future reasoning.

Phase II makes memory less lonely

After MemCells are formed, EverMemOS performs Semantic Consolidation. New MemCells are embedded and assigned to the nearest MemScene if similarity passes a threshold; otherwise, a new MemScene is created.

That sounds like clustering, because it is. But the purpose is not just tidier storage. The point is to create a higher-level memory object that can accumulate related episodes over time.

A single MemCell may say the user measured a waist circumference of 104 cm. Another later MemCell may say it is now 96 cm. A third may say the user’s weight has remained stable. Separately, these are health-related snippets. Consolidated, they form a trajectory: the user is tracking progress, has improved a metric, and may prefer actionable guidance for continuing the routine.

The paper’s profile example uses exactly this kind of pattern. EverMemOS maintains a compact user profile with explicit facts and implicit traits, updated from scene summaries rather than individual turns. That matters because profile systems are especially vulnerable to stale or overconfident memory. A profile should not blindly freeze a user as “overweight,” “beginner,” “price-sensitive,” or “prefers X” forever. It should know which facts are stable, which changed, and which are supported by recent evidence.

This is also where conflict handling enters the story. The paper describes recency-aware updates and conflict tracking in the profile module. For business systems, that is not a cosmetic detail. If an account manager’s assistant remembers both “client prefers annual billing” and “client asked to switch to monthly billing after cash-flow pressure,” the memory system needs to surface the updated state without deleting the history that explains it.

Flat retrieval can retrieve both facts. It cannot automatically decide which one defines the present.

Phase III retrieves by reconstruction, not rummaging

The third phase is Reconstructive Recollection. The phrase is slightly academic, but the mechanism is concrete.

Given a query, EverMemOS first retrieves relevant MemCells using hybrid dense retrieval and BM25 over atomic facts, fused with Reciprocal Rank Fusion. It then scores MemScenes through their most relevant MemCells, selects a small set of scenes, pools candidate episodes, reranks them, and filters foresight by time validity.

Then comes the interesting control step: an LLM-based sufficiency checker decides whether the retrieved context is enough to answer the query. If not, the system rewrites the query to search for missing information.

This is not just retrieval. It is retrieval with a feedback loop.

The appendix gives a representative LoCoMo multi-hop trace. The question asks whether James lives in Connecticut. The first retrieval round finds related information but lacks explicit residence confirmation, so the sufficiency checker marks the context insufficient. The system then generates refined queries about James’s residence and Connecticut. The second round retrieves evidence that James adopted a dog from a shelter in Stamford, which supports the final answer. The answer is judged correct by all three LLM judges.

The example matters because it shows what “agentic retrieval” means in this paper. It does not mean the agent theatrically thinks harder. It means the retrieval controller identifies missing evidence and searches again with better queries. A small miracle: the system admits it does not have enough evidence before answering. Someone alert enterprise AI governance; this behavior is suspiciously healthy.

The authors report that on LoCoMo with GPT-4.1-mini, the sufficiency checker triggers a second-round query rewrite for 31.0% of questions. That is a useful operational signal. If roughly one-third of difficult memory questions need a second retrieval pass, then single-shot retrieval should not be treated as the default ceiling for memory systems.

The main results say structure helps where scattered evidence hurts most

The main experiments evaluate EverMemOS on LoCoMo and LongMemEval, with a profile study on PersonaMem-v2.

LoCoMo contains 1,540 questions over 10 ultra-long dialogues of roughly 9,000 tokens each, including single-hop, multi-hop, temporal, and open-domain questions. LongMemEval’s S-setting contains 500 questions over much longer conversations of roughly 115,000 tokens each. The authors compare against Zep, Mem0, MemOS, MemoryOS, and MemU, while standardizing answer-generation backbones where possible.

The headline results are strong, but the distribution of gains is more informative than the overall score.

On LoCoMo with GPT-4.1-mini, EverMemOS reaches 93.05% overall accuracy, compared with 85.22% for the strongest baseline, Zep. The biggest improvements appear in categories that require connecting evidence across time:

The pattern is exactly what the mechanism predicts. If a question depends on a single fact, many systems can retrieve it. If the question depends on dispersed evidence, changing states, or temporal relations, scene-level consolidation becomes more valuable.

On LongMemEval, EverMemOS reaches 83.00% overall accuracy, compared with 77.80% for MemOS. Again, the most interesting gain is not everywhere equally. The paper reports a +20.6% relative improvement on knowledge-update tasks, where the system must handle changed information. It also improves single-session assistant and multi-session categories. But it does not win every subcategory: it is lower than MemOS on single-session preference tasks and tied on temporal reasoning.

That is worth keeping in the article, because it prevents the usual paper-summary disease: “new method better, limitations later, applause now.” The actual evidence says EverMemOS is especially strong when the problem resembles its design target: scattered, evolving, cross-session memory. It is not magic dust sprinkled over all memory questions.

The ablations are the paper’s most useful evidence

The ablation study is more valuable than the headline table because it tests whether the architecture’s pieces matter.

The authors remove parts of EverMemOS and measure how performance changes. On LoCoMo, full EverMemOS scores 93.05%. Removing MemScenes drops performance to 89.16%. Removing MemCells and relying on raw dialogue retrieval drops it further to 81.82%. Removing external memory collapses performance to 0.52%. LongMemEval shows the same direction: full EverMemOS reaches 83.00%, without MemScenes 79.60%, without MemCells 71.20%, and without external memory 5.00%.

The interpretation is clean:

This is where the paper’s argument becomes more persuasive. The performance does not come only from using a better retriever or a larger prompt. It degrades stepwise as memory structure is removed. That is exactly what one would expect if the architecture’s hierarchy is doing real work.

The segmentation test is also useful, but for a narrower reason. The authors compare semantic segmentation against fixed-message chunks, token chunks, and session boundaries. Under the isolated setting used for the test, semantic segmentation performs better than fixed heuristics and even beats session-level oracle boundaries. Results are also similar when using GPT-4.1-mini or Qwen3-4B for segmentation.

This should be read as a robustness check, not a second thesis. It suggests the system is not absurdly dependent on perfect episode boundaries. It does not mean every enterprise should immediately trust an LLM to segment all business memory without monitoring. We are not running a theology department.

The cost story is not “free memory”; it is “selective memory”

The paper includes a cost analysis, and it complicates the story in a useful way.

EverMemOS performs LLM-mediated operations during memory construction and retrieval. On LoCoMo with GPT-4.1-mini, Phase I memory construction consumes 9.42 million tokens. Phase III retrieval plus answer generation consumes 10.27 million tokens, about 6.7k tokens per question. With GPT-4o-mini, the comparable numbers are 9.34 million for construction and 9.31 million for retrieval plus answering, about 6.0k tokens per question.

So no, structured memory is not free. It moves cost around.

The paper’s better claim is that EverMemOS can occupy a favorable accuracy-efficiency frontier by retrieving a compact number of scenes and episodes. The default setup retrieves 10 MemScenes and selects 10 Episodes. Sensitivity analysis shows gains saturating around 10 scenes, and the episode-count frontier shows that moderate retrieval budgets can outperform strong baselines without simply stuffing more text into the answer prompt.

For business use, this matters because memory costs arrive in two places:

1. Write-time cost: building MemCells and maintaining scenes/profiles.
2. Read-time cost: retrieving, checking sufficiency, rewriting queries, and answering.

The business question is therefore not “Is EverMemOS cheaper?” The better question is: Which memory operations can be amortized, cached, batched, or run asynchronously, and which must happen at response time?

For a customer support assistant, many memory updates can happen after the conversation. For a trading assistant, clinical-adjacent coach, or compliance-sensitive workflow, some retrieval verification may need to happen synchronously. Same architecture, different cost tolerance. The spreadsheet will be less poetic than the paper, but more decisive.

The profile study points to personalization, but the absolute numbers stay modest

The PersonaMem-v2 results test whether consolidated profiles help. In the ablation table, using both episodic evidence and the user profile reaches 53.25% overall accuracy, compared with 48.30% for profile-only and 43.93% for episodes-only. In the appendix’s full comparison, EverMemOS also beats the strongest baseline, MemOS, 53.25% vs. 50.72%.

That is a meaningful improvement. It is also a reminder that personalized memory remains hard. A result in the low 50s is not a license to ship autonomous life coaching with a confident voice and a pastel dashboard.

The more useful interpretation is qualitative: profile and episode memory are complementary. Episodes preserve evidence. Profiles summarize stable patterns. A system that only stores episodes may miss the user trajectory. A system that only stores profiles may lose grounding and become a personality horoscope wearing an API key.

The best version uses both:

Episodes answer: "What happened?"
Profiles answer: "What has this tended to mean about the user?"
Foresight answers: "What temporary or future-facing constraint matters now?"

This division maps well to enterprise applications. A CRM copilot needs account events, account profile, and upcoming obligations. A tutor needs learning episodes, learner profile, and scheduled goals. A healthcare-adjacent assistant needs reported symptoms, longitudinal metrics, and temporary constraints. The core business object may change, but the memory logic stays similar.

The case studies show what benchmarks still miss

The paper’s qualitative cases are not main evidence. They are better read as exploratory demonstrations of behaviors current benchmarks do not fully test.

In one case, the system recalls that a user’s previous injury was a Grade-II right ankle sprain during badminton, rather than giving a generic explanation about sports injuries. In another, it uses a longitudinal health trajectory, such as waist reduction and stable weight, to suggest a next-stage goal. In the travel case, it remembers that the user previously had a bad experience with overcrowding and missing advance tickets in Beijing, then applies that experience to a future Europe trip by recommending advance reservations and off-peak visits.

These cases illustrate a gap between benchmark accuracy and useful agent behavior. Benchmarks often ask whether the system can answer a memory question. Real products ask whether the system can use remembered experience at the right moment without overgeneralizing it.

That is the more difficult frontier. A system that remembers a failed Beijing trip and suggests advance museum booking is helpful. A system that turns one crowded trip into “this user hates all popular destinations forever” is personalization by caricature. EverMemOS gives a structure for better memory use; product teams still need policies for scope, consent, sensitivity, and decay.

What Cognaptus would infer for business systems

The paper directly shows that structured memory improves performance on long-horizon conversational-memory benchmarks, especially multi-hop, temporal, and knowledge-update tasks. It also shows that removing memory structure degrades accuracy, and that profile information adds signal beyond episodic retrieval.

The business inference is broader but should be stated carefully: many enterprise AI systems should treat memory as a governed lifecycle, not a retrieval add-on.

A practical architecture inspired by EverMemOS would look like this:

This makes the ROI argument less fluffy. The value is not “the agent remembers more.” The value is lower repeated discovery cost, fewer context resets, better escalation summaries, safer personalization, and more consistent handling of changing constraints.

But there is a hard boundary: this paper does not prove production ROI. It does not test real enterprise deployment, privacy operations, adversarial memory poisoning, organizational permission boundaries, or multi-user account memory where different people from the same company say conflicting things. Those are implementation problems, not footnotes.

Still, the direction is important. The next serious memory systems will not look like a vector database with nicer labels. They will look more like a small operating system for experience: write policies, update policies, retrieval policies, sufficiency checks, conflict tracking, expiry rules, and audit trails.

Where the evidence should not be overread

EverMemOS is a strong paper, but its evidence has clear boundaries.

First, the evaluation is text-only. The authors note that MemCell and MemScene abstractions are modality-agnostic, but multimodal and embodied settings are outside the scope. That matters for agents operating over screenshots, voice, documents, dashboards, and physical-world signals.

Second, the system uses LLM-mediated operations for memory construction and retrieval. The paper argues that caching, batching, and asynchronous processing can help, but end-to-end efficiency remains future work. In production, latency and cost will determine whether the full lifecycle runs continuously, selectively, or only for high-value users and workflows.

Third, current benchmarks do not fully stress ultra-long timelines. A benchmark conversation of 115k tokens is large, but a real customer relationship, student history, or employee support trail can span years, systems, teams, and policy changes. Memory decay, retention, deletion, and provenance become governance requirements.

Fourth, personalization raises privacy risk. The paper’s profile mechanism is exactly the kind of component that can create value and legal exposure at the same time. Explicit facts, implicit traits, and time-varying measurements must be handled with consent, visibility, and deletion controls. Otherwise, “the agent knows me” becomes “why does the agent know that?” which is a less charming user experience.

Memory becomes useful when it stops being a pile

EverMemOS is best understood as a shift from memory as storage to memory as organized experience.

The system’s three phases are not decorative architecture. MemCells stabilize noisy episodes. MemScenes consolidate related experience. Reconstructive recollection assembles only the context needed for a query and checks whether retrieval is sufficient before answering. The benchmark gains, ablations, sensitivity tests, profile study, and case examples all point to the same lesson: structure matters most when the evidence is dispersed, time-sensitive, or in conflict.

That is also the business lesson. Enterprise AI does not fail only because it lacks data. It fails because past interactions sit in separate piles: chat logs, CRM notes, tickets, emails, documents, profiles, and dashboards. Retrieval can find pieces. Useful memory has to organize them into a state the model can reason over.

The junk drawer is not memory. It is clutter with confidence.

EverMemOS shows what happens when an AI system starts cleaning the drawer before it reaches inside.

Cognaptus: Automate the Present, Incubate the Future.