Sparse Routing May Buy You Inspectability, Not Just Efficiency

TL;DR for operators

Herbst, Wermter, and Lee find that the analyzed Mixture-of-Experts models often represent tested concepts in far fewer neurons than comparable dense transformers.¹ The difference is largest under the hardest probe constraint: when only one neuron is available, MoE experts often approach their own best probe performance while dense feed-forward layers need more dimensions. Models with sparser routing also tend to show cleaner representations.

That changes the interpretability question. If a model’s architecture already contains relatively coherent experts, an audit team may not always need to reconstruct useful features entirely after training. The paper shows a workflow for labeling whole experts, validating those labels on held-out examples, checking whether the labeled expert contributes to predicted outputs, and measuring how narrowly experts specialize. The result is promising for debugging and targeted diagnosis, but it is evidence for relative inspectability, not proof that experts are fully monosemantic or that natural-language labels constitute causal or governance controls.

Sparse routing changes what has to be reconstructed after training

A dense transformer creates an awkward inspection problem: one internal unit can respond to several unrelated concepts. That makes neuron-level explanations unstable, and motivates post-hoc methods that reconstruct more interpretable features from dense activations.

MoE architectures introduce another constraint. A router sends each token to only a subset of experts. The standard reason for doing this is computational: a model can have large total capacity without activating all of it for every token.

The paper asks whether that routing decision also changes representation quality. The proposed mechanism is straightforward. An expert receives a narrower distribution of inputs because the router filters which tokens reach it. If that restricted input distribution also reduces pressure to combine unrelated features inside the same neurons, then sparsity could produce cleaner internal representations before any separate interpretability method is applied.

The authors call the combination of cleaner internal features and router-filtered inputs modular monosemanticity. The evidence supports “more monosemantic than comparable dense representations,” not “perfectly monosemantic.”

The single-neuron gap is the core result

The main experiment compares 12 pretrained dense and MoE language models across 58 concepts drawn from part-of-speech, LaTeX, code, and ordinary text. For each concept, the authors train probes restricted to $k$ selected neurons, with $k$ ranging from 1 to 64.

The key quantity is not merely whether a concept can be decoded. It is how many neurons are required before the probe performs well. If one neuron performs nearly as well as a larger set, the concept is represented more compactly.

Across the analyzed models, MoE expert activations outperform active-parameter-matched dense feed-forward activations at low $k$, with the largest architecture gap at $k=1$. MoE experts often reach near-optimal or near-perfect probe performance with a single neuron. The effect appears across all four concept categories rather than being confined to one dataset type.

The paper also examines whether this could simply reflect parameter count. A within-family comparison between OLMoE-1B-7B and dense OLMo-7B is particularly informative: the sparse model still performs better on sparse probing even though the dense model uses substantially more active parameters per token. This is a robustness check against a simple capacity explanation, not randomized proof that routing sparsity itself caused the difference.

The cross-model pattern points in the same direction. Mixtral-8x7B, the densest MoE in the study, is less interpretable under these measures than sparser MoEs, while Qwen3-30B-A3B is among the cleanest. The reasonable interpretation is therefore an association between greater routing sparsity and lower measured polysemanticity, with architecture and training differences still preventing a clean causal estimate.

Whole experts become plausible units of interpretation

Cleaner neurons alone would not justify treating an entire expert as a coherent module. The paper therefore moves up one level.

For selected experts, the authors collect 32-token sequences in which the expert produces a large router-weighted update to the residual stream. An external LLM receives 20 high-impact examples together with tokens promoted by the expert and proposes a natural-language description of its behavior. A separate scorer then evaluates that description on positive examples from the target expert and negative examples from other experts in the same layer.

Most interpreted experts in the selected OLMoE, ERNIE, and Qwen3 layers receive label F1 scores above 0.8, with Qwen3 frequently above 0.9.

The examples also show why “expert” should not be read as “subject-matter department.” Representative behaviors include chemical and biological suffixes, syntactic coordination, patent and legal citation patterns, and closing LaTeX environments. The architectural module can be narrower than a domain and more structured than a keyword detector.

Attribution makes the labels stronger, not complete

A label that predicts high-activation examples could still be descriptive correlation. The authors therefore test whether the labeled expert contributes to the output behavior implied by the description.

They select experts from OLMoE layers 4, 9, and 14, generate manually checked trigger-target cases, and rank same-layer experts by their direct contribution to the target token. On matched prompts, the focal expert is usually among the strongest contributors and is consistently routed. On control prompts, the same expert has little attribution and is not routed in 80% of cases.

This is materially stronger evidence than label agreement alone. It links the proposed expert function to the model’s output computation.

It is still attributional evidence. The study does not systematically ablate experts, randomly intervene on routing, or establish that the label captures every context in which the expert matters. The authors’ documented low-F1 cases show why: descriptions can overfit frequent examples, a particular semantic domain, or an unnecessarily strict syntactic trigger.

For operators, the right unit is therefore an attribution-supported hypothesis, not a certified module specification.

Deep experts look increasingly like task modules

The paper’s specialization analysis sharpens this interpretation. Instead of asking only which inputs route to an expert, it separately measures what kinds of output tokens the expert promotes.

Vocabulary tokens are clustered in the model’s output-embedding space at resolutions from 10 broad clusters to 5,000 fine-grained clusters. Expert specialization is then measured as the divergence between an expert’s cluster distribution and the layer-wide base rate, with a simulated random-expert baseline subtracted to control finite-sample noise.

Several deeper experts become much more specialized as cluster granularity increases. For OLMoE-L15-E17, a LaTeX-closing expert, functional specialization rises from 0.049 at $k=10$ clusters to 0.349 at $k=5000$, while routing specialization remains comparatively weak. OLMoE-L14-E59 shows a similar pattern, reaching 0.417 functional specialization at the finest resolution.

That distinction matters. Some experts may receive fairly generic inputs yet perform highly specific output transformations. In later layers, the better mental model is often a narrow computational sub-circuit rather than a broad topic router.

What model teams can use now—and what remains research

The paper directly supports a narrower claim than “MoEs are interpretable.” In the models tested, sparse routing is associated with cleaner low-dimensional representations, and existing experts can often be described at a useful functional level.

Cognaptus inference begins from there. Architecture teams evaluating MoE designs may want to treat routing sparsity as a possible inspectability property alongside throughput, memory, and quality. Model-auditing teams could use expert routing, high-impact activations, and attribution patterns to narrow investigation to a smaller architectural unit before escalating to more expensive neuron- or feature-level analysis. Debugging a formatting failure, domain-specific completion error, or suspicious transformation may become cheaper if the responsible computation is already localized.

Three boundaries prevent that from becoming a governance claim. The study does not randomize routing sparsity during training, so it cannot isolate a causal treatment effect. Natural-language labels remain dependent on the explainer model, prompts, and sampled examples. And direct attribution is not the same as proving through intervention that changing or removing an expert will safely change only the intended behavior.

The largest contemporary MoE systems were also excluded for compute reasons, and early-layer functional interpretation is less reliable because the Logit Lens projection used by the analysis is noisier there.

The credible next step is therefore targeted validation: ablation, counterfactual routing, out-of-distribution prompts, and tests of whether expert-level interventions remain localized under real deployment conditions.

Inspectability may be partly architectural

The paper’s most consequential result is not the automatic labels. It is the possibility that interpretability cost is partly determined before an interpretability team begins its work.

Sparse routing appears to change which inputs an expert sees and how compactly tested concepts are represented inside it. That can leave the trained model with modules that are easier to inspect directly. The expert-labeling and specialization analyses then show what becomes possible once that structure exists.

For model builders, this introduces a new architecture question: not only how many parameters can be activated efficiently, but whether the routing design leaves the resulting computation easier to localize and test.

The present evidence is strong enough to justify measuring that property. It is not yet strong enough to treat expert labels as controls.

Cognaptus: Automate the Present, Incubate the Future.