Follow the Heads, Not the Hype: How LLMs Route Deductive Reasoning

A compliance bot does not fail only when it gives the wrong final answer.

It can fail earlier, in a quieter and more expensive place: it selects the wrong premise, stops collecting evidence too soon, matches the wrong rule, and then writes a perfectly fluent explanation of a decision that was already broken three steps ago. Very elegant. Very useless.

That is why the paper Revealing Algorithmic Deductive Circuits for Logical Reasoning is more interesting than another “LLMs can reason” or “LLMs cannot reason” entry in the usual scoreboard ritual.¹ The paper does not mainly ask whether a model gets deductive reasoning questions right. It asks where, inside the model, the reasoning procedure appears to be routed.

The answer is not a single magic neuron, and not the whole model “thinking hard.” The authors identify sparse groups of attention heads that appear to mediate specific sub-tasks in symbolic-aided deductive reasoning: reading facts, reading rules, selecting premises, deciding whether premise selection should stop, selecting a rule, and implementing the traversal strategy implied by few-shot demonstrations.

For business readers, the practical message is not that we can now open any commercial model and inspect its logic module like an engine part. We cannot. The paper works with explicit symbolic reasoning formats, open models, synthesized counterfactual prompts, and mechanistic interpretability tools. The useful lesson is narrower and more operational: when AI systems make rule-governed decisions, final-answer accuracy is a blunt instrument. The fragile part may be a small intermediate decision that routes the rest of the reasoning chain.

The paper turns reasoning into a graph traversal problem

The paper studies deductive logical reasoning under a symbolic-aided chain-of-thought format. Instead of asking the model to produce a loose natural-language explanation, the prompt represents reasoning as a structured inference process over facts and rules.

A simplified reasoning chain looks like this:

KB = {A, K, F}
=> F(KB['A'], Rule4) => 'D'
=> F(KB['F', 'K'], Rule2) => 'E'
=> Validate(KB, Question='E') = True

This is not ordinary prose. It is closer to an execution trace. The model must maintain a knowledge-base snapshot, choose which known facts can be used, match them against rule conditions, select the right rule, derive a new fact, and continue until the query can be validated.

That matters because the reasoning process can be decomposed. Once decomposed, it can be attacked scientifically. The authors are not merely asking, “Did the model answer True or False?” They ask which token positions steer the reasoning path and which internal components causally affect those positions.

The setup also explains why the accepted article structure here should be mechanism-first. If we summarize the paper by saying “the authors found reasoning heads,” we flatten the main contribution. The paper’s value is the pipeline: uncertain steering tokens are identified, controlled prompt pairs are created, activation patching localizes responsible heads, path patching maps information flow, and ablation tests whether those heads matter outside the synthetic discovery setting.

The costly decisions are not the syntax tokens

The first useful observation is almost embarrassingly practical: not every token in a reasoning trace is equally difficult.

In the preliminary experiment, the authors categorize tokens by reasoning role and inspect token probabilities across several models, including Llama-3.1-8B-Instruct, Qwen3-8B, Phi-4, and Qwen3-4B. Tokens with probability below 0.8 are treated as uncertain. The exact threshold is not a law of nature, but it gives the authors a way to identify where the model is least confident while generating the gold reasoning chain.

The uncertain positions concentrate around three reasoning components:

This is the paper’s first conceptual move. The hard part of chain-of-thought reasoning is not necessarily writing the template or copying the symbolic notation. Syntax can be cheap. Steering is expensive.

In business terms, this distinction is familiar. A workflow agent may always produce a well-formatted approval memo. That does not mean it used the correct policy clause. A contract review assistant may always cite “Section 8.2.” That does not mean Section 8.2 was the right rule for the factual situation. Format compliance is not reasoning compliance. It is a nice costume, but still a costume.

Controlled corruption makes the circuit search testable

The paper’s methodology depends on clean-corrupted prompt pairs.

For each reasoning component, the authors synthesize pairs of prompts that share the same broad structure but differ in a causal element that should change the target reasoning decision. For example, they may corrupt a fact so that premise selection should change, corrupt rule content so that rule selection should change, or change the traversal algorithm in the demonstrations so that premise selection should follow a different search strategy.

This design has a clear purpose: it makes activation patching meaningful.

Activation patching asks a counterfactual question. If we run the model on a corrupted prompt, then restore the activation of a specific attention head from the clean run, does the model recover the clean behavior at the target token? If restoring one head strongly moves the logit toward the clean target, that head is treated as causally important for that component.

The authors then use path patching to estimate information flow between pairs of important heads. This second step is crucial. A list of heads is not yet a mechanism. Path patching asks which heads transfer information to which other heads, allowing the authors to describe a circuit network rather than a loose collection of hot spots.

Here is the clean way to read the experiments:

This table matters because papers like this are easy to over-read. The authors are not saying they have found “the reasoning module” of all LLMs. They are showing that, under a structured symbolic reasoning setup, a small subset of attention heads has measurable causal importance for distinct reasoning roles.

That is already enough. No need to inflate it into enlightenment.

Reading heads appear earlier; decision heads appear later

The paper’s most business-relevant mechanism is the separation between reading and deciding.

The authors find that causal information-reading heads, such as heads that read facts or rule conditions, tend to appear earlier than decision-making heads. This pattern is reported across evaluated models. The interpretation is straightforward: lower and middle layers retrieve or route local factual and rule information, while later components integrate that information to make the next reasoning move.

That is not surprising, but it is useful. A deductive trace is not one undifferentiated blob. It has stages:

read relevant facts/rules
        ↓
match rule conditions
        ↓
select premise and rule
        ↓
decide whether the current inference step is complete
        ↓
continue traversal until validation

The authors report a consistent temporal computational structure: matching rule conditions comes early, implementing the traversal algorithm follows, then premise and rule selection, and then premise termination. The exact layer positions differ by model, but the staged pattern is the point.

For enterprise AI, this suggests a useful design metaphor. A reasoning audit should not only ask whether the model reached the correct final state. It should instrument the route:

1. Did the model read the right evidence?
2. Did it match the right rule condition?
3. Did it choose the correct rule?
4. Did it stop too early or continue unnecessarily?
5. Did the final answer depend on a valid chain, or did the model merely preserve the output format?

The fifth question is where many demonstrations look better than they are. The model can keep the symbolic costume intact even after the logic underneath has failed. The paper explicitly notes that, after ablation, some final-answer accuracy on ProntoQA and ProofWriter appears attributable to random guessing after incorrect reasoning chains. That is the kind of result a business dashboard usually hides, because dashboards adore final columns and dislike intermediate shame.

Sparse heads can carry large effects

One of the more striking findings is sparsity.

For rule selection, the authors report that a small number of heads account for a dominant proportion of the causal effect. In Llama-3.1-8B-Instruct, the highest average indirect effect score exceeds 30%. In Qwen models, peak scores exceed 12%.

The important point is not the exact number alone. It is the asymmetry. Some reasoning decisions appear to depend disproportionately on a few heads, especially when the decision has become relatively deterministic after earlier premises have already constrained the search space.

This has two interpretations.

First, it is good news for mechanistic interpretability. If reasoning-relevant effects were smeared uniformly across the whole model, circuit discovery would become much harder. Sparse high-impact heads give researchers something to localize, ablate, and compare.

Second, it is bad news for naive robustness assumptions. If a small set of components disproportionately affects rule selection, then small internal disruptions may have large behavioral consequences. That does not mean production systems should start “patching heads” next quarter. It means we should stop treating reasoning failures as if they were always vague semantic misunderstandings. Some failures may be localized routing errors: the model had the relevant information available but moved the wrong piece of it into the next step.

In a compliance setting, that difference is not academic. A model that lacks the required policy document has a retrieval problem. A model that has the policy document but chooses the wrong premise has a reasoning-control problem. The remediation strategy is different.

Circuit networks are modular, but not cleanly single-purpose

The paper’s circuit-network analysis adds a useful complication: heads can be polysemantic.

Some attention heads participate in multiple reasoning sub-tasks. The authors report that Llama-3.1-8B-Instruct tends to share heads primarily for causal reading roles, while Qwen models show more sharing across decision-making roles. In other words, the broad mechanism is similar, but the allocation of sub-tasks is architecture- or model-specific.

This is exactly the kind of detail that prevents the paper from becoming a cartoon.

The mechanism is modular in the sense that the authors can identify role-associated heads and sub-circuits. But it is not modular in the software-engineering sense, where one clean function handles one clean task with a polite docstring and a unit test. Attention heads may route several kinds of information. Decision heads may integrate multiple upstream signals. Different models may reuse heads differently.

For business readers, the translation is simple: do not expect interpretability to give you a neat organizational chart of the model’s “departments.” It may give you something messier but still useful: a map of recurring causal bottlenecks.

A useful enterprise analogy is process mining. A process-mining tool does not always reveal a clean official workflow. It reveals the actual paths taken by cases, including loops, shortcuts, and shared bottlenecks. Mechanistic interpretability is doing something similar here, but inside the transformer.

Ablation is where the paper earns its claim

The ablation experiments are the paper’s strongest validation step.

After identifying logical reasoning heads on synthesized data, the authors knock out top heads and test performance on the synthesized benchmark, ProntoQA, ProofWriter, and MMLU. They compare this against random head ablation. The configurations distinguish between rule-selection-related heads, premise-selection-related heads, premise-termination-related heads, and a combined three-role ablation.

The purpose is not just to damage the model and announce that damage happened. The purpose is to test whether the discovered heads are actually necessary for reasoning behavior, and whether that necessity extends beyond the synthetic discovery setup.

The results support three readings:

The ProntoQA and ProofWriter detail is especially important. The paper notes that, after ablation, some remaining final-answer accuracy can be explained by random guessing: ProntoQA has two possible answers, while ProofWriter has three. In other words, a model can preserve the surface form of the reasoning trace while losing the meaningful reasoning process.

That is a serious warning for evaluation. If a system produces a chain-of-thought-shaped explanation and lands on the right binary answer, the final score may overstate reasoning quality. A coin can also be correct. It just has terrible documentation.

The business value is cheaper diagnosis, not mystical transparency

The practical value of this paper is not that companies can immediately inspect every deployed model’s attention heads. Most enterprise AI users work with hosted models, closed weights, toolchains, RAG layers, orchestration frameworks, and governance constraints. They cannot casually run activation patching on a vendor API.

The real value is conceptual and methodological: the paper gives a better way to think about reasoning evaluation.

For rule-governed business applications, the question should shift from:

Did the answer look right?

to:

Where in the reasoning route can the answer first go wrong?

That shift supports a more useful evaluation framework:

This does not require every company to perform mechanistic interpretability. It requires companies to stop treating “reasoning” as a black-box score and start treating it as a sequence of inspectable control points.

For many Cognaptus-style automation projects, that is the difference between a demo and an operational system. A demo can impress with fluent explanations. An operational system needs failure localization. When the system fails, the business needs to know whether the defect came from retrieval, rule matching, premise selection, tool execution, or final synthesis. Otherwise every failure becomes “the AI was wrong,” which is emotionally satisfying and operationally empty.

What the paper directly shows, and what we should infer carefully

The paper directly shows that, under a symbolic-aided CoT format for deductive reasoning, attention heads associated with specific reasoning roles can be localized using causal mediation analysis. It also shows that knocking out those heads damages reasoning performance more than random ablation, including on ProntoQA and ProofWriter.

Cognaptus’ business inference is that AI reasoning evaluation should become more component-level. If a system is expected to handle policy checks, contract interpretation, compliance triage, underwriting rules, or workflow approvals, the evaluation should include tests for intermediate reasoning decisions, not only final-answer accuracy.

The uncertain part is transfer.

The paper’s reasoning traces are explicit, symbolic, and structured. Many business documents are not. Real contracts contain vague definitions, exceptions, cross-references, jurisdictional context, negotiated amendments, and human ambiguity. Enterprise workflows also include retrieval systems, structured databases, APIs, user permissions, and changing policy versions. The paper does not solve all of that. It was not trying to.

The proper conclusion is therefore bounded:

• The paper is strong evidence that deductive reasoning in LLMs can involve sparse, role-specialized attention-head circuits under controlled symbolic prompting.
• It is good evidence that some circuit roles generalize from synthesized data to established logical reasoning benchmarks.
• It is not evidence that any arbitrary business reasoning answer can be trusted because the model writes a good chain of thought.
• It is not a ready-made production diagnostic for closed commercial models.
• It is a useful research signal for building better reasoning audits, stress tests, and intervention methods.

That boundary is not a weakness. It is what keeps the paper useful instead of theatrical.

The bigger lesson: reasoning reliability needs route inspection

The common misconception is that chain-of-thought quality is mostly a surface prompt-format issue. Add “think step by step,” ask for a structured explanation, maybe sprinkle in symbolic notation, and the model will reason more faithfully.

The paper pushes against that view.

The symbolic format matters, but the mechanism is not just the format. The model must internally route facts, rules, and traversal strategy through a sequence of constrained decisions. Some of those decisions are fragile. Some are handled by sparse, high-impact heads. Some heads read; others decide; some do both. Remove the wrong set, and the reasoning trace can remain grammatically alive while logically dead.

That is the line worth carrying into business AI.

A reasoning system should not be trusted because it sounds deliberate. It should be trusted when its route can be tested: evidence selected, rules matched, intermediate decisions validated, final answer derived. The paper gives researchers a mechanistic version of that principle. Businesses can adopt the evaluation version now.

The next generation of enterprise AI quality control will not be satisfied with “the answer was correct on the sample set.” It will ask where the system looked, what it selected, which rule it matched, why it stopped, and whether the final answer actually followed.

Less theater. More trace.

A shocking proposal, apparently.

Cognaptus: Automate the Present, Incubate the Future.