Mind the Gaps: Why LLMs Reason Like Brilliant Amnesiacs

A model can write a flawless explanation, check its own work, announce a correction, and then make the same mistake three paragraphs later. This is the familiar enterprise horror show: the AI appears to reason, but its reasoning has no working memory of its own commitments. It is articulate, capable, and sometimes genuinely useful. It is also, in the wrong setting, a brilliant amnesiac.

That is the useful way to read Cognitive Foundations for Reasoning and Their Manifestation in LLMs, a large study of how reasoning behaviours appear in model traces across text, image, and audio tasks.¹ The paper does not waste time on the sterile question of whether LLMs “really reason”, which is where productive conversations go to retire. Instead, it asks a more operational question: which cognitive behaviours appear in model reasoning, how are they sequenced, and when do those structures actually correlate with success?

The answer is uncomfortable for anyone still treating chain-of-thought as a universal solvent. The problem is not simply that models lack reasoning behaviours. Often, they display plenty of them. The problem is that they deploy the wrong ones for the problem type, in the wrong order, with too little adaptation when the task becomes ambiguous.

In other words: not no reasoning. Misallocated reasoning. Very enterprise.

The failure mode is structural, not theatrical

The paper’s central move is to stop judging reasoning by its outward theatricality. Long explanations, self-checks, numbered steps, and confident verification can all look reassuring. But a model can perform these behaviours while still failing to construct the right representation of the problem.

That distinction matters because different problems demand different cognitive structures.

A well-structured task has clear goals, defined constraints, and verifiable answers. Algorithmic problems, rule-following tasks, and many story problems fall into this region. These tasks tolerate linear processing reasonably well. You can proceed step by step, check local consistency, and eventually land on the answer.

Ill-structured tasks behave differently. Diagnosis, case analysis, design, and dilemmas do not hand the model a neat path. The goal may be ambiguous. The solution may require trade-offs. The evaluation criteria may conflict. These problems demand scoping, representation selection, domain alignment, abstraction, and meta-cognitive monitoring before the answer-building begins.

The paper’s mechanism-first insight is that models often reverse this requirement. They bring broad cognitive behaviour to tasks where it is less necessary, then narrow into rigid sequential processing precisely when ambiguity makes flexibility valuable. That is the gap.

A model does not fail because it forgot to “think step by step”. It fails because step-by-step is sometimes the wrong cognitive costume for the party.

The paper builds a vocabulary for reasoning traces

To make this measurable, the authors synthesise cognitive-science theories into a taxonomy of 28 cognitive elements grouped into four dimensions:

This taxonomy is not the article’s punchline. A taxonomy-first summary would be a labelled filing cabinet, and nobody needs another filing cabinet with “cognition” written on the drawer. The value is in what the taxonomy enables: trace-level diagnosis.

The authors annotate reasoning traces at span level, identifying where these cognitive elements appear in generated reasoning. They then examine not only whether elements are present, but how they are arranged. A trace can contain verification and still use it badly. It can include decomposition after it has already committed to the wrong framing. It can backtrack without learning anything from the backtrack. This is where the amnesia starts looking less metaphorical.

The empirical design is large, but the interpretation is behavioural

The study analyses 192,709 reasoning traces from 18 models across text, audio, and image settings, plus 54 human think-aloud traces used as qualitative reference points. The text models include open-weight reasoning systems such as Qwen3 variants, DeepSeek-R1-Distill variants, Olmo 3 thinking models, OpenThinker, s1.1, DeepScaleR, DeepHermes, and DeepSeek-R1. The audio side uses Qwen3-Omni-30B-A3B-Thinking. The image traces come from Zebra-CoT, which includes curated and frontier-model-refined multimodal reasoning traces rather than raw commercial-model thought processes.

That last detail matters. The authors exclude commercial models such as GPT-4 and Gemini from the main raw-reasoning-trace analysis because those systems expose summarised traces, not necessarily the underlying reasoning process. Sensible. Annoying for benchmarking completeness, but sensible.

The task taxonomy extends Jonassen’s problem-solving framework into 13 problem categories. These range from factual recall and logical problems through algorithms, story problems, rule-using tasks, decision-making, troubleshooting, diagnosis-solution, case analysis, design, and dilemmas, with creative or expressive tasks added for cases that do not fit goal-directed problem solving.

Correctness is assessed with GPT-4o as judge, using ground truth for verifiable tasks and a strong reference response for non-verifiable tasks. Cognitive elements are annotated using GPT-4.1 after human-in-the-loop prompt refinement. The reasoning structures are represented as transition graphs, with edges capturing sequential, parallel, or containment relationships between cognitive-element spans. Success-associated structures are extracted using NPMI-based weighting.

That sounds technical because it is. The business translation is simpler: the paper tries to convert “the model gave a weird answer” into “the model skipped the cognitive moves normally associated with success on this class of problem”.

That is a useful diagnostic upgrade.

Models use the broadest reasoning where they need it least

The first major result is a deployment mismatch. Models do not simply underuse cognitive elements everywhere. On well-structured problems, they often show broad behavioural engagement. On ill-structured problems, they narrow their repertoire.

That is backwards.

For algorithmic, story, and rule-using problems, models frequently display many cognitive elements. But as problems become more ill-structured and less verifiable, traces concentrate around sequential organisation, logical coherence, and forward chaining. Those are not useless. They are just insufficient when the task requires constructing a representation before marching through it.

The successful traces tell a different story. For ill-structured problems, success correlates with more diverse representations and operations: hierarchical organisation, network organisation, spatial or temporal organisation, backward chaining, representational restructuring, pattern recognition, and other non-linear behaviours. The harder the problem is to pin down, the more the model needs to decide how to think about it before generating the answer.

This is where the common reader misconception breaks. More chain-of-thought is not automatically better reasoning. More verification is not automatically better reasoning. More steps are not automatically better reasoning. A model can produce a long, polished failure because it is optimising the appearance of procedure while missing the structure of the problem.

The paper’s examples make this concrete. In logical reasoning, a human participant quickly abstracts the key invariant of a checkerboard-domino problem: each domino covers one black and one white square, so removing two same-coloured corners makes coverage impossible. DeepSeek-R1, by contrast, produces thousands of tokens, repeatedly counts, checks, reconsiders, and eventually circles toward the relevant idea. It is not that the model never verifies. It verifies plenty. The issue is that verification becomes motion rather than memory.

In a design-style healthcare policy problem, the human trace shows strategy selection, source assessment, ordinal ranking, evaluation, and explicit surprise when a calculated result conflicts with expectation. The model trace is shorter and more fluent, but it leans into generic causal explanation and policy synthesis. It sounds like a briefing note. That is charming, provided one does not mistake charm for disciplined reasoning.

Sequence matters because late cognition is often decorative

The paper’s second major contribution is to show that cognitive elements are not Lego bricks scattered on the floor. Their placement matters.

For algorithmic problems, the authors compare common reasoning structures against structures associated with successful traces. Some frequently used elements, including self-awareness and backtracking in the common pattern, carry negative NPMI scores in that context. Translation: models often perform behaviours that look reflective, but in that setting those behaviours correlate with failure.

For diagnosis-solution problems, the contrast is sharper. Successful traces follow a scoping pattern: selective attention, sequential organisation, knowledge alignment, and then forward chaining. In practical terms, they identify relevant features and align them to domain constraints before building the solution.

The common pattern bypasses that scoping phase and rushes directly into forward chaining. The paper reports a common forward-chaining node probability of 0.748 in this diagnosis setting. That is the machine-learning version of a consultant entering a room and immediately recommending “a platform strategy”. We have all seen this movie. It was not improved by slides.

This matters because cognitive behaviours can be performative when they arrive too late. Decomposition after a wrong frame has already been selected may simply organise the wrong answer. Verification after the model has committed to a shallow representation may only verify local consistency inside a bad map. Backtracking without remembering why a path failed becomes expensive hesitation.

The structure, not just the inventory, determines whether the reasoning helps.

Humans are not just shorter; they choose different abstractions

The human comparison is deliberately modest. The paper uses human think-aloud traces as qualitative reference points, not as a definitive benchmark of human cognition. That boundary is important: 54 human traces are useful for contrast, not enough to settle a species-level argument. We can put away the evolutionary trumpets.

Still, the contrast is informative.

Humans show stronger presence of abstract behaviours such as self-awareness and abstraction. They often invoke conceptual processing earlier, especially in logical tasks. They may leave many intermediate steps implicit, which makes their traces shorter but not necessarily less structured. LLMs, by contrast, externalise more steps, use more backward-chaining-like moves in some cases, and repeat verification or backtracking without necessarily incorporating the result into subsequent reasoning.

That pattern supports the article’s title. The model is not stupid. It is not even lazy. It is often highly capable at producing local reasoning moves. But it can behave as if each move fails to update a stable strategic state. It checks, then forgets what the check implied. It backtracks, then repeats the explored path. It notices uncertainty, then proceeds as though uncertainty were an ornamental phrase.

Brilliant amnesiac is not a diagnosis. It is an operational risk category.

The steering experiment is promising, but not a magic prompt spell

The paper then turns the analysis into an intervention. The authors extract successful cognitive structures by problem type and automatically convert them into test-time guidance prompts. This is not hand-crafted prompt artistry. It is an automated pipeline that takes consensus structures and scaffolds the model toward the behaviours associated with success for that class of problem.

The evaluation uses roughly 50 textual problems per model and problem type, balanced between previously correct and incorrect answers. That design tests two things at once: can guidance recover failures, and does it avoid breaking cases the model already solved?

The result is heterogeneous, which is exactly what makes it credible. Capable reasoning models, especially Qwen3-family models and larger R1-Distill variants, show substantial gains on complex open-ended categories such as diagnosis, case analysis, and dilemmas. The reported gains reach up to 66.7% in a model-problem cell, with other strong gains such as +60.0% on dilemmas for Qwen3-14B and R1-Distill-Qwen-32B.

But weaker models can degrade badly. DeepScaleR-1.5B and Hermes-3-Llama-3-8B show losses exceeding 50% in several categories. Design problems are even more awkward: the paper reports catastrophic failure after steering and excludes them from the steering table.

So the result is not “add cognitive scaffolding and enjoy free intelligence”. The result is more precise: capable models may possess behavioural repertoires associated with success but fail to deploy them spontaneously. Structural guidance can help elicit those behaviours, especially on ill-structured problems. Below a capability threshold, however, scaffolding may overload or constrain the model.

That is a useful boundary. It prevents the paper from becoming another prompt-engineering fairy tale, which the industry already has in bulk packaging.

What the paper directly shows

The paper directly supports four claims.

That final column is not academic throat-clearing. It is the part an enterprise buyer should read twice.

The business value is diagnosis before delegation

For business use, the paper’s value is not that it gives executives a new way to praise “human-like reasoning”. Please no. The value is that it points toward reasoning diagnostics before delegation.

Most enterprise AI evaluation still asks: did the model get the answer right on the test set? That remains necessary, but it is too thin for agentic workflows. A model may pass a benchmark while using brittle structures that collapse under ambiguity, novelty, or changing constraints. In regulated or high-cost domains, this becomes expensive quickly.

A Cognaptus-style operational pathway would look like this:

1. Classify the task structure before selecting the model behaviour.
2. Identify which cognitive elements are usually associated with success for that task class.
3. Inspect traces or intermediate artefacts for missing, misplaced, or performative behaviours.
4. Route the task to a model, scaffold, or human reviewer based on cognitive risk.
5. Treat reasoning structure as part of assurance, not as decorative explanation after the fact.

The strongest business fit is not arithmetic automation. It is high-ambiguity work: claims triage, legal case analysis, compliance interpretation, medical-adjacent research support, incident diagnosis, investment memo review, policy design, and complex customer escalations. These are tasks where the model must choose a representation, not merely apply a rule.

For these workflows, “accuracy on a benchmark” is not enough. The more useful question is: did the system perform the cognitive moves that this kind of problem requires?

A practical framework for enterprise evaluation

The paper suggests a simple but powerful evaluation shift: from answer scoring to reasoning-structure profiling.

This is not a call to expose all chain-of-thought to users. In production, raw traces may be unavailable, unsafe to reveal, or unreliable as literal internal explanations. The practical target is not necessarily public reasoning transcripts. It is internal diagnostic instrumentation: structured scratchpads, intermediate plans, checklists, verifier outputs, retrieval choices, and decision logs that reveal whether the system selected an adequate cognitive pathway.

The paper gives a language for that instrumentation.

The current research field is overtraining the easy-to-measure parts

One of the paper’s more revealing side analyses examines 1,598 LLM reasoning papers and asks which cognitive elements the research community emphasises. The imbalance is exactly what one would expect from a field addicted to measurable progress: context awareness, decomposition and integration, knowledge alignment, sequential organisation, pattern recognition, and abstraction receive much more attention than self-awareness, spatial organisation, temporal organisation, ordinal organisation, and evaluation.

This is not because the neglected elements are unimportant. The paper’s empirical findings suggest that diverse behavioural repertoires matter especially for ill-structured problems. The neglected behaviours are simply harder to benchmark, harder to reward, and less compatible with the industry’s favourite ritual: announce a number, put it on a leaderboard, pretend the number is a map.

The business implication is straightforward. If vendors optimise what is easiest to measure, buyers inherit blind spots where measurement is hardest: ambiguity, self-assessment, representation switching, and strategy adaptation. Those are also the places where enterprise workflows become risky.

The awkward coincidence is left as an exercise for procurement.

Boundaries that matter for practical use

The paper is valuable, but it should not be overread.

First, the framework measures observable behavioural traces. It does not prove that a model has internal cognitive mechanisms equivalent to human reasoning. A trace can be produced by genuine structured processing, learned stylistic imitation, cached patterns, or some hybrid nobody can fully inspect from the output alone.

Second, the dataset composition is uneven. Some problem types are underrepresented. Strategic performance has no instances in the analysed dataset. Creative or expressive tasks appear only minimally. Text has broader problem-type coverage than audio or image. This affects how confidently one should generalise across modalities and categories.

Third, the human comparison is small and qualitative. It is useful for illustrating differences in trace structure, not for declaring a comprehensive human-versus-machine theory of mind. The participants are educated adults using think-aloud protocols, sometimes with tools. That is an interesting comparison group, not humanity in a jar.

Fourth, the correctness evaluation for non-verifiable tasks depends on reference responses and LLM judging. That is a reasonable scalable method, but it inherits the usual uncertainty around open-ended judgement. In business settings, the reference standard may need to come from domain experts, policy rules, or outcome data.

Fifth, the steering intervention is not universally safe. It helps some capable models on some ill-structured tasks and harms weaker ones. The design-problem failure is particularly important: a scaffold that looks theoretically principled can still collapse in practice. Cognitive guidance should be tested like any other intervention, not worshipped as a new incantation.

Finally, real commercial deployment often lacks access to faithful reasoning traces. Many production models provide summaries, not full internal reasoning. Enterprises therefore need diagnostic proxies: structured plans, tool-use logs, retrieval justifications, consistency checks, task classifiers, and external verifiers.

The paper opens a path. It does not pave the road, install lighting, and hand over a maintenance contract.

The real lesson: reasoning is task routing all the way down

The most useful business lesson is not that LLMs need more reasoning. It is that reasoning itself must be routed.

A well-structured problem can tolerate linear procedures. A diagnosis problem needs scoping before solving. A design problem needs goal management and trade-off representation. A dilemma needs context awareness, value conflict handling, and evaluation under ambiguity. Treating all of these as “just ask the model to think harder” is not strategy. It is vibes with an API key.

The paper makes a strong case for replacing generic chain-of-thought optimism with cognitive structure management. For enterprise AI, that means evaluating whether the model is using the right reasoning behaviour for the task, not whether it is producing enough reasoning-shaped text to calm the room.

LLMs are not empty parrots, and they are not reliable junior analysts trapped in silicon. They are systems with uneven behavioural repertoires, strong local fluency, brittle deployment habits, and surprising responsiveness to the right scaffold. Their failures are often less like ignorance than like mis-sequenced competence.

That is annoying. It is also actionable.

The next generation of AI evaluation should not merely ask whether a model can answer. It should ask whether the model knows which kind of thinking the problem deserves.

For now, many models still reason like brilliant amnesiacs: capable enough to impress, forgetful enough to scare, and fluent enough to make the forgetting sound intentional.

Cognaptus: Automate the Present, Incubate the Future.