Pieces, Not Puzzles: How ArcMemo Turns LLM Reasoning into Reusable Skills

Tickets repeat. Spreadsheets repeat. Compliance reviews repeat. Code reviews repeat. Not exactly, of course. That would be merciful. They repeat with just enough variation to make last month’s solution almost useful and therefore mildly dangerous.

This is where many enterprise “AI memory” systems become filing cabinets with delusions of competence. They store prior chats, snippets, tickets, documents, and summaries, then hope the next prompt will rhyme closely enough with something in the archive. Sometimes it does. Often it does not. The agent remembers the old puzzle, not the transferable piece.

ArcMemo, introduced in ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memory, attacks that boundary directly.¹ Its central claim is not that LLMs need more memory. That claim has already been overfed. The sharper claim is that reasoning systems need concept memory: abstract, modular, reusable patterns distilled from verified solution traces, then selectively retrieved and composed for new problems.

That distinction sounds academic until an agent starts carrying around every prior workflow as if context length were strategy. It is not. A bigger backpack does not make the hiker wiser; it just makes the fall more expensive.

ArcMemo’s mechanism: write concepts, select pieces, compose solutions

ArcMemo is easiest to understand as a loop:

1. Solve a problem.
2. Extract reusable concepts from the successful trace.
3. Store those concepts in an external memory.
4. For a new problem, select only the concepts that look relevant.
5. Use those concepts to guide a fresh solution.
6. When feedback confirms a new solution, update memory again.

The important verb is extract. ArcMemo is not trying to preserve a full transcript of prior reasoning. It is trying to turn a successful reasoning trace into smaller building blocks that can survive a change of surface form.

In the paper’s ARC-AGI setting, each task is a small pixel-grid puzzle. The model must infer the transformation rule from example input-output pairs and apply it to test grids. One puzzle may involve counting objects, another may involve extending a pattern, another may involve identifying symmetry, and a harder puzzle may require composing several such ideas. ARC-AGI is useful here because it punishes simple memorisation. The tasks look small, but the generalisation problem is rude.

ArcMemo’s design decomposes memory into three surfaces:

That third surface is where many memory systems quietly fail. Storing useful ideas is not enough. The system must retrieve the right ones without dumping the entire attic into the working context. ArcMemo treats selection as part of the reasoning process, not as an afterthought bolted to a vector database with hope and a cosine score.

This is not generic RAG, and that is the point

The likely misreading of ArcMemo is to file it under “RAG for reasoning.” Understandable. Also too blunt.

Classic retrieval-augmented generation retrieves documents, facts, passages, or examples. ArcMemo retrieves concepts. A retrieved item is not “here is the old puzzle and its answer.” It is closer to “when objects must be sorted by a derived property, use a routine parameterised by the property extractor.” That is a different object. It is not evidence to quote; it is a skill to reuse.

The paper tests two memory formats.

The first, Open-Ended (OE), stores natural-language situation-suggestion pairs. It is deliberately light: under condition X, consider doing Y. This is easy to write and flexible, but it leaves more structure to the model.

The second, Program-Synthesis (PS), is stricter. It represents concepts as typed, parameterised entries inspired by software reuse. A PS concept can be a type, structure, or routine. It includes fields such as title, description, parameters, output type, relevance cues, and implementation notes. The use of parameters matters because it lets one stored idea cover many concrete variants. The use of output typing matters because it hints at how concepts can be composed.

That is the paper’s most useful design move. It borrows a lesson from software engineering: reusable components are not just shorter descriptions. They have interfaces.

Abstraction helps only if selection can keep up

There is a tension inside concept memory. Make a concept too specific, and it becomes easy to apply but hard to reuse. Make it too abstract, and it travels well but may fail to latch onto the new case. ArcMemo does not magically remove this tension. It designs around it.

For OE memory, selection uses a preprocessing step. A vision-language model captions the puzzle into structured observations, then the system matches those descriptions against stored situation fields. This is a lightweight way to translate spatial input into memory-searchable text.

For PS memory, selection becomes more deliberate. The paper argues that standard embedding retrieval is too “fast intuition” for highly abstract concepts. A concept may be relevant even when the current puzzle shares few lexical or surface features with prior cases. ArcMemo-PS therefore uses reasoning-based selection: the model explores the puzzle, identifies candidate concepts using relevance cues, and attempts to fill parameters by using type annotations and related concepts.

This is slower than embedding lookup. It is also the kind of slowness that may be necessary when the memory item is no longer a document snippet but a reusable reasoning component. Cheap retrieval is useful only when it retrieves the right thing. Otherwise it is just a fast way to be confidently irrelevant, a popular enterprise software genre.

What the experiment actually tests

The paper evaluates ArcMemo on a randomly selected 100-puzzle subset of the ARC-AGI-1 public validation split. The authors seed memory using 160 manually authored Python solutions from the public training split, previously created in related ARC-AGI work. They use o4-mini as the main reasoning model and GPT-4.1 for auxiliary tasks such as abstraction and non-reasoning selection.

The evaluation uses a program-synthesis setup. Instead of asking the model to directly output the final grid, the system asks for a transformation function. This allows generated code to be tested against the reference input-output pairs, creating a feedback signal for retries and memory updates. That choice matters: ArcMemo assumes some form of test-time feedback is available. Without feedback, memory can preserve mistakes with admirable consistency.

The results should be read in layers:

That last distinction is important. The paper does not say embeddings are useless. It says a tested embedding-based approach performed poorly in this setting, apparently over-indexing on lexical similarity and retrieving irrelevant or overly broad concepts. The practical conclusion is not “never use embeddings.” It is “do not assume semantic search is enough when the thing being retrieved is an abstract skill.”

The headline number is useful, but not the whole story

Under the standard official scoring setup, the no-memory o4-mini baseline scores 55.17. ArcMemo-PS scores 59.33. That is a 7.5% relative gain over the baseline. ArcMemo-OE scores 56.67, while the cheatsheet-style memory baseline scores 57.67.

The gain is not gigantic. Good. Gigantic gains on small, expensive benchmark slices usually deserve suspicion before applause. The more interesting point is that ArcMemo-PS is the only tested memory design that consistently beats the no-memory baseline across all tested inference scales.

With one retry, ArcMemo-PS reaches 67.33 versus the baseline’s 66.67. With two retries, it reaches 70.83 versus the baseline’s 69.00. The advantage narrows as more compute is spent, which is exactly what one should expect. If the model gets more chances to rediscover ideas from scratch, memory’s marginal benefit shrinks. Memory is most valuable when it prevents rediscovery, not when brute-force exploration has already done the job.

There is also a useful embarrassment in the comparison with the cheatsheet baseline. At two retries, the cheatsheet setting scores 71.33, above ArcMemo-PS’s 70.83. But the cheatsheet is not consistently above the baseline across scales; at one retry it falls below the no-memory baseline. This makes ArcMemo-PS less of a universal winner and more of a steadier design. For production systems, steadiness is not glamorous. It is merely the thing procurement eventually notices.

Selection is not housekeeping; it is part of intelligence

The selection ablation is one of the paper’s most business-relevant parts. Removing selection from ArcMemo-PS reduces the official score at standard compute from 59.33 to 55.17. With one retry, selected PS memory scores 67.33 versus 66.00 without selection. With two retries, selected memory scores 70.83 versus 70.00 without selection.

The paper also reports that the no-selection setting uses far more tokens. This matters because memory is often sold as a cost reducer: retrieve the old idea, save reasoning effort. ArcMemo complicates that story. Memory-augmented runs can increase output token usage because retrieved concepts encourage the model to explore more hypotheses. In other words, memory can make the model think more, not less. Tragic, but sometimes useful.

For builders, the lesson is blunt: memory without selection is not memory architecture. It is prompt inflation.

The qualitative analysis strengthens the point. When comparing ArcMemo-PS without selection to the cheatsheet-style cumulative memory setting, the authors find that both solve the same number of puzzles across all runs, but differ on ten puzzles. In the cheatsheet setting, only 40% of new solves could be linked to memory elements in the generated cheatsheet; for ArcMemo-PS without selection, 100% of new solves could be linked to concept memory contents. The authors are careful not to claim direct causality, because LLM interpretability remains a swamp with nicer typography. Still, the analysis suggests ArcMemo’s modular concepts are more plausibly connected to the solves they enable.

Continual learning appears, but only under the right conditions

The paper’s continual-learning experiment is narrower than the headline may imply. It is run on ArcMemo-OE, not the PS variant, and updates memory every ten problems during evaluation. The effect appears mainly at higher retry depth.

At zero retries, continual updating does not help: official score moves from 56.67 to 56.00. At one retry, it also does not improve the official score: 65.67 versus 65.00. At two retries, it does help: 67.67 rises to 70.00.

That pattern is plausible. Continual memory needs successful new traces before it can write useful concepts. More retries create more solved examples; solved examples create more memory updates; updated memory can then help later attempts. The benefit is not instant magic. It is a compounding loop that needs verified wins to feed it.

This is exactly where enterprise analogies should be handled carefully. A customer-support agent could store reusable troubleshooting routines after tickets are resolved. A code agent could store refactoring or debugging patterns after tests pass. A finance agent could store reconciliation routines after workpapers validate. But those systems need trustworthy feedback signals. “The model said it looked right” is not feedback. It is vibes with JSON.

The business lesson: design the memory object before buying the memory store

Most enterprise memory discussions start with infrastructure: vector database, graph database, long-context model, archive policy, retention settings. ArcMemo suggests starting one layer above that: what kind of thing deserves to be remembered?

For agentic workflows, the best unit may not be a document, a chat, a ticket, or a full trajectory. It may be a typed routine with relevance cues and implementation notes.

This does not mean every enterprise should implement ArcMemo literally. ARC-AGI pixel puzzles are not insurance claims, procurement disputes, or bank reconciliations. The useful transfer is architectural: store verified abstractions, expose interfaces, select only relevant pieces, and update memory when feedback justifies it.

That last clause does a lot of work. In ARC-AGI, feedback is conveniently available through input-output examples and program execution. In business domains, feedback may be delayed, noisy, political, or only partially observable. A medical diagnosis workflow cannot safely update its memory because an early note “felt useful.” A compliance agent cannot treat a passed internal draft as proof that a regulatory interpretation is correct. Memory hygiene is not an implementation detail. It is the difference between organisational learning and automated superstition.

Where the evidence stops

ArcMemo is promising, but the boundaries are sharp.

First, the evaluation is on a 100-task subset of ARC-AGI-1 public validation, not the full public validation set and not ARC-AGI-2. The authors justify this by cost and sampling variance, but the boundary remains. The absolute gain should be read as evidence of a mechanism, not as a universal benchmark conquest.

Second, the memory is seeded with existing human-authored solutions from public training puzzles. This is reasonable for the experiment, but it means the system is not starting from an empty organisational memory. In business terms, ArcMemo is closer to a system initialised with expert playbooks and then refined through experience than a blank agent that learns everything on the job. Frankly, that is how serious systems should be built anyway.

Third, retrieval quality remains unresolved. The paper’s own embedding-retrieval experiment performed poorly, while reasoning-based selection costs more. That creates a practical design trade-off: cheap selectors may miss abstract relevance; deep selectors may improve relevance but consume budget. No free lunch, just a different invoice.

Fourth, continual updates introduce order dependence. If solving task A creates a memory that helps task B, then the sequence of tasks matters. Batching improves throughput but may reduce the chance for immediate learning. Production systems will need update schedules, consolidation passes, and audit trails. Otherwise “lifelong learning” becomes “whatever happened first this morning.”

Finally, abstraction itself is not settled. The appendix’s small specificity experiment found that highly detailed concepts performed better on a tiny validation subset, while lower-specificity versions degraded. That is not a contradiction of ArcMemo’s thesis; it is the central engineering tension. Useful memories must be abstract enough to transfer and specific enough to act. Anyone promising both automatically has probably confused a demo with a discipline.

The practical pattern: concept memory as operational capital

ArcMemo’s value is not that it turns LLMs into self-improving geniuses. Spare us. Its value is that it gives a concrete pattern for converting verified work into reusable operational capital.

A good enterprise ArcMemo-like system would have five layers:

1. Trace capture: record the reasoning artefacts that led to a successful outcome.
2. Verification gate: write memory only when tests, reviewers, audits, or outcomes provide sufficient signal.
3. Concept abstraction: convert the trace into a reusable object with parameters, relevance cues, and implementation notes.
4. Selective retrieval: choose a small set of relevant concepts for each new task.
5. Consolidation: periodically merge, prune, and restructure concepts so the memory library does not become a museum of near-duplicates.

The most important part is not the model call. It is the schema. If the memory object is badly shaped, no database will rescue it. If the organisation stores entire incidents rather than reusable procedures, it will retrieve entire incidents. Then the agent gets to cosplay as an intern reading old tickets at speed.

ArcMemo’s mechanism-first contribution is therefore broader than ARC-AGI. It reframes memory as interface design for reasoning reuse. The paper’s numerical results support that framing, but the architecture is the part builders should steal with dignity.

Conclusion: memory should carry skills, not souvenirs

The reason ArcMemo is interesting is not that it adds another memory module to the agent stack. The stack has plenty of modules. Some of them even have names that survived the pitch deck.

ArcMemo is interesting because it asks what memory is supposed to preserve. Not the whole prompt. Not the whole trace. Not a vague summary of prior experience. The reusable piece.

On ARC-AGI-1, that design improves o4-mini’s official score from 55.17 to 59.33 under standard compute, stays above the no-memory baseline across tested inference scales, benefits from selection, and shows limited but suggestive evidence of continual improvement when memory is updated from verified solves. The result is not a blank cheque for self-learning enterprise agents. It is a blueprint with conditions.

For businesses, the translation is simple enough to be uncomfortable: if your AI systems repeat reasoning work, their memory should look less like a transcript archive and more like a library of tested, typed, selectable skills. Store the pieces. Compose the next puzzle.

Cognaptus: Automate the Present, Incubate the Future.