AI Memory

Memory is cheap until it has to remember correctly. A chatbot can remember a paragraph for a few minutes. An enterprise agent is asked to remember a customer’s old address, current address, account owner, exception approval, product issue, refund promise, and the reason the promise changed last month. Then it must answer without mixing the past with the present. This is where “just add more context” begins to look less like strategy and more like buying a bigger drawer for unsorted receipts. ...

Memory is usually sold as a storage problem. Give the agent a vector database. Add a recall layer. Save summaries. Search harder. Expand the context window until the budget department starts making eye contact. Then ask the agent a simple question: what changed after the earlier conversation? That is where the polite demo often turns into a fog machine. ...

Memory is easy to sell and hard to govern. Every enterprise AI demo eventually reaches the same theatrical moment: the agent remembers something. A prior customer preference. A workflow exception. A formatting habit. A failed action that should not be repeated. Everyone nods. Someone says “continuous learning.” A roadmap slide appears. The slide is almost certainly too optimistic. ...

Upgrade. That is the moment when the usual agent-memory story starts to look too small. Imagine a company has run a long-term AI assistant for six months. It has managed client context, learned internal workflows, developed preferences for how reports should be structured, tracked unresolved decisions, and built a working relationship with several humans. Then the platform upgrades the underlying model. ...

A customer tells your AI assistant that she dislikes crowded tourist attractions. Three weeks later, she asks for a weekend itinerary. A good assistant should not proudly recommend the busiest landmark in the city. A less good assistant will do exactly that, but in a warm tone. This is the quiet failure mode behind many “personal AI” demos. The interface remembers the conversation. The product claims continuity. The model may even have a giant context window large enough to swallow a small novel. Yet when the user asks a new question, the system behaves as if the earlier preference is just decorative text floating somewhere in the attic. ...

Memory fails in a very ordinary way. A customer asks, “Can we use the same approval condition as before?” A research agent says, “Yes.” A procurement assistant retrieves the old vendor quote. A planning copilot remembers a hotel price from yesterday’s itinerary. Everything looks semantically relevant. The words match. The entities match. The embedding score smiles politely. ...

Memory is useful until nobody can find where it lives. That, in miniature, is the operational problem with today’s language models. They can answer questions, imitate expertise, retrieve fragments of the past, and produce very confident nonsense with the composure of a senior consultant who has just discovered bullet points. But when a model gives a wrong factual answer, the organisation deploying it faces an awkward question: where, exactly, is that wrong fact stored? ...

TL;DR for operators Long-context windows are useful. They are also an expensive way to pretend that memory is just a bigger clipboard. The HEMA paper argues for a more operationally realistic design: keep a compressed summary of the conversation always visible, store detailed past exchanges outside the prompt, and retrieve only the details that matter for the current turn.1 That gives the model two different memory behaviours: continuity from Compact Memory and factual recall from Vector Memory. ...