Stable Enough to Be Wrong: Why Neuron Selectors Need Causal Audits

TL;DR for operators

A team using an importance-ranking tool must know whether its low-ranked model components are genuinely safe to remove—not merely whether the tool produces the same ranking repeatedly.

On LLaMA-3.1-8B at 30% masking, Wanda achieved near-perfect ranking stability of 0.9998, yet removing the components it labelled least important drove perplexity to 360,083. Consensus-2 was less stable at 0.945, but the same removal test produced perplexity of 66.1. The more reproducible ranking was far less causally faithful.

The paper audits selectors through direct intervention: it removes selected model output components without retraining, then compares the damage caused by deleting supposedly unimportant components with the damage caused by deleting supposedly important ones. Attribution-based selectors separate these groups more reliably than magnitude-, activation-, and Wanda-based alternatives across the tested models.

The same logic is applied to refusal behavior by identifying components that respond differently to matched harmful and benign prompts, then checking whether their removal changes harmful-prompt refusal without unacceptable benign over-refusal or general capability loss. Layer-matched random controls show that the identities of the selected components—not merely their layer locations—drive the effect.

For pruning, interpretability, and safety-editing teams, stability should remain a reproducibility signal rather than an approval criterion. A selector should pass direct removal tests, row-identity controls, utility constraints, and validation on the target architecture before it guides an expensive pruning pipeline or a sparse behavior edit.

A reproducible ranking can still be operationally wrong

Any team pruning or editing a model eventually faces the same decision: which internal components can be altered without removing capabilities the system still needs?

Importance scores appear to provide an answer. A selector ranks model components, engineers remove the low-ranked ones, and recovery training repairs the remaining damage. In interpretability work, the same ranking may be presented as evidence that a small collection of components carries a behavior.

A common validation shortcut is reproducibility. If a selector produces nearly identical rankings across calibration samples, the ranking looks dependable.

The results in Faithfulness to Refusal: A Causal Audit of Neuron Selectors challenge that inference directly.¹ On LLaMA-3.1-8B at 30% masking, Wanda achieved a Spearman rank correlation of 0.9998 across calibration subsets. Yet removing the rows it classified as least important raised WikiText-2 perplexity to 360,083. Consensus-2 was less stable, with a correlation of 0.945, but its corresponding perplexity was 66.1.

The first ranking repeated more consistently. The second was far better at identifying rows the model could lose.

The audit asks selectors to survive direct removal

The paper replaces proxy validation with a controlled structural intervention. Each candidate component is one output row in an attention or MLP linear layer. Removing it means setting that row and its associated bias to zero while leaving the rest of the checkpoint unchanged.

No fine-tuning, recovery procedure, or inference-time hook is added. This isolates the selector from the optimization machinery that might otherwise compensate for a poor ranking.

The audit then performs two matched tests:

• Least-Relevant-First: remove the rows the selector says are least important.
• Most-Relevant-First: remove the rows it says are most important.

A faithful selector should produce limited damage in the first test and substantially greater damage in the second. The paper summarizes this separation as:

Both tests use the same model, masking rate, dataset, and intervention. The variable that changes is row identity. This makes the gap more informative than perplexity after a pruning-and-recovery pipeline, where retraining can conceal whether the original selector was accurate.

Across five base models containing roughly 377,000 to 3.5 million prunable rows, attribution-based methods—Layer-Wise Relevance Propagation, Integrated Gradients, and their Consensus-2 rank aggregation—generally identify dispensable rows two to four orders of magnitude more accurately than non-attribution baselines. They also inflict stronger damage when their highest-ranked rows are removed.

At 30% masking, Consensus-2 produces the lowest least-relevant-first perplexity on four of the five models and matches Integrated Gradients on the fifth. On larger models, the attribution methods also preserve more downstream benchmark utility.

Stability measures repeatability, not causal correctness

The stability analysis is not a secondary benchmark. Its purpose is to test whether a widely available validation signal predicts the result of intervention.

It does not.

Mean activation is another instructive case. On LLaMA-3.1-8B, it achieves a Spearman correlation of 0.994 and a 30% mask Jaccard overlap of 0.934. Those numbers imply that calibration subsets produce nearly the same ranking and select nearly the same rows. But removing its supposedly low-importance rows raises perplexity to 139,874.

LRP is less stable, with a Spearman correlation of 0.913 and mask overlap of 0.635, yet its most-relevant-first removal causes far more damage than its least-relevant-first removal. Its ranking is noisier across calibration subsets but more causally discriminating.

This does not make stability irrelevant. A selector whose rankings change arbitrarily would be difficult to operate. But stability answers whether a procedure repeats itself. It does not establish that the repeated ranking corresponds to the components carrying the measured capability.

The paper’s consensus controls sharpen this point. Genuine LRP–IG agreement improves identification of safe-to-remove rows, while rank-randomized aggregation does not reproduce the benefit. Yet disagreement is not merely noise: veto variants that retain rows favored by either attribution method outperform Consensus-2 on both removal directions for LLaMA-3.1-8B and Qwen3-8B. Cross-method disagreement can contain causal signal that averaging suppresses.

Refusal editing tests behavior-specific faithfulness

The audit becomes more consequential when the target is not general language modeling but refusal behavior.

The paper constructs a refusal-versus-compliance logit margin and attributes it separately on matched harmful and benign prompts. Rows are favored when they promote compliance more strongly on harmful prompts than on their benign counterparts. A penalty then down-ranks rows that are important for general language modeling.

This is a behavior-specific test rather than a search for components that merely activate strongly on safety-related text.

For LLaMA-3.1-8B, the resulting LRP masks raise harmful-prompt refusal to 0.814 for hate and 0.926 for crime in the layer-matched evaluation. The selected operating points remove 2% of rows and keep benign refusal and perplexity within the paper’s feasibility constraints.

The critical evidence comes from the control. The researchers generate random masks with exactly the same number of removed rows in every layer as the real attribution mask. Only the row identities are randomized.

Those controls achieve mean refusal rates of 0.274 for hate and 0.264 for crime. Across all five domains, the real masks outperform the layer-matched control means by 0.455 to 0.765.

Layer placement therefore does not explain the intervention. The selected row identities carry behavior-specific causal information.

The experiment establishes sufficiency: removing these rows is enough to increase refusal. It also establishes specificity relative to masks with the same layer allocation. It does not show that these rows are required for refusal or that they form its only valid implementation.

Multiple row sets can support the same behavior

A single universal selector does not emerge from the cross-model results.

Integrated Gradients dominates most domains on the two smaller LLaMA-3.2 models. LRP and Integrated Gradients split the domains on LLaMA-3.1-8B and Qwen3-8B. On Qwen3-8B, for example, LRP produces higher refusal for hate and crime, while Integrated Gradients performs better for adult and legal prompts.

Architecture changes which attribution path recovers the more effective intervention.

The successful LRP and Integrated Gradients masks also overlap surprisingly little. Their top 1% row sets share only about 3% to 6% of rows across models and domains, even though both can install refusal.

The supported interpretation is not that one method must have found the authentic circuit and the other failed. Refusal appears to occupy a redundant, architecture-dependent subspace containing multiple sparse row sets sufficient to alter behavior.

The masks are nevertheless structured. Refusal-relevant rows concentrate in MLP gate and up projections. Harm categories also form overlap clusters, especially hate with crime and medical with legal. Contrastive signal density is strongest for hate and crime, which generally require smaller masks. Medical and legal often need 10% masking and can cross into substantial capability damage.

Validation gates for compression and safety teams

The paper directly evaluates selectors, not organizational processes. Cognaptus draws three operational implications from its evidence.

The financial value is avoided misallocation. A team can reject a selector before spending compute on recovery training, extensive red-teaming, checkpoint packaging, or production integration.

For safety edits, the same gates should precede deployment claims. A high harmful-prompt refusal rate is not enough when the same mask raises benign over-refusal, damages reasoning benchmarks, or succeeds only under one refusal judge.

The evidence supports diagnosis, not deployment

The audit covers dense decoder-only models from the LLaMA-3, Qwen3, and Gemma-3 families. It does not test mixture-of-experts systems, encoder-decoder architectures, multilingual behavior, or several other attribution and intervention families.

Most selector evaluations use one run at a fixed operating point. Cross-architecture replication and the 70 model-selector-domain cells provide breadth, but they do not replace repeated-seed uncertainty estimates.

The refusal margin and the evaluation classifier may both respond to recognizable refusal language. Transfer to SorryBench reduces the likelihood that the masks only exploit the CAST prompts, but surface-form mimicry remains unresolved.

Most importantly, stronger edits can produce substantial OR-Bench over-refusal or collapse general utility, particularly in lower-signal medical and legal domains. The edited checkpoints are diagnostic artifacts, not evidence of production-ready alignment.

Causal faithfulness must be tested on the target model

The paper’s main contribution is not another neuron-scoring formula. It is an acceptance test.

A selector that looks stable, computationally efficient, or compatible with downstream recovery may still rank the model’s components incorrectly. Direct removal reveals whether the ranking separates rows the model can lose from rows whose absence changes capability or behavior.

For operational teams, the replacement standard is concrete: matched interventions for sufficiency, layer-matched controls for specificity, utility constraints for acceptable cost, and architecture-specific replication before trust.

Reproducibility can tell an organization that its selector will make the same decision again. A causal audit tests whether it is the decision the model can survive.

Cognaptus: Automate the Present, Incubate the Future.