Mechanistic Interpretability

TL;DR for operators A model can distinguish two behavioral concepts accurately without providing a reproducible internal measurement of either. In From Stories to Features, Scherbakov and colleagues1 address this problem by extracting concept directions from large collections of matched synthetic stories. Their 1,036-feature dictionary supports natural-text retrieval, controlled behavioral steering, and exploratory analysis of agent failures and jailbreak attacks. ...

TL;DR for operators When inspecting a vision model, the sparse autoencoder that best reconstructs its internal activations may not produce the clearest explanation of what image regions those activations represent. Spatial-SAE addresses a specific modeling limitation: conventional sparse autoencoders do not explicitly coordinate features across neighboring image patches. Its two-stream representation improves concept recovery and spatial alignment in the studied settings, but generally sacrifices reconstruction fidelity. ...

TL;DR for operators An enterprise AI assistant may retrieve two related facts correctly when asked separately, yet fail to combine them into one answer. Research on looped language models shows that later processing passes can repair this specific failure. Transplanting later-loop attention queries into an earlier pass rescued 89.8% and 96.9% of selected synthetic errors in two Ouro models, substantially outperforming control interventions. ...

TL;DR for operators An AI evaluator may report separate judgments about grammar, vocabulary, and coherence without applying those criteria independently. In a study of 25 base language models, interventions targeting the rubric that had been degraded produced only 0.0074 more expected score points than interventions targeting other rubrics—just 1.74% of the absolute matching effect. The difference was statistically detectable but far below the researchers’ practical significance threshold. ...

TL;DR for operators A conversational AI assistant may correctly identify a character’s emotions in a benchmark while remaining unreliable when a customer expresses frustration or concern indirectly. In ToMStories, the strongest open-weight model achieves 88.4% multiple-choice accuracy, above a small human baseline of 80.3%. Yet the highest model score on open-ended questions is 58.6%, below the human baseline of 64.2%. ...

TL;DR for operators Audio AI can retain information about a speaker’s voice and still produce an answer that follows a contradictory written description. In controlled experiments, audio language models favored conflicting textual claims at a reported mean rate of 78%, rising to 86% when sarcasm and pacing were excluded. The research traces this failure to how textual information influences downstream representations, rather than simply to missing acoustic information. A targeted attention intervention substantially reduced text dominance during free-form generation in one model, but the benefit depended on intervention placement and evaluation format. ...

TL;DR for operators A hybrid model can contain relatively little attention and still depend on that attention for its decisive retrieval step. Across 23 measured depth profiles from twelve checkpoints, the intermediate clue needed for a two-hop answer becomes most causally usable immediately before the layer that performs the largest conversion toward the final answer. That conversion occurs in an attention-containing layer in every measured profile. ...

TL;DR for operators An internal probe can be highly accurate without providing a reliable way to change model behavior. In Xining Xun’s study of language-model development,1 internal probe readability was already near ceiling at the earliest 1,000-step checkpoint across all six Pythia sizes. Yet interventions that pushed activations along those same probe directions were statistically indistinguishable from matched random-direction edits in 43 of 48 model-by-checkpoint cells. ...

TL;DR for operators A translation can be wrong in at least three different ways: the model can arrange words incorrectly, produce the wrong language, or choose the wrong lexical content. The paper examined here finds evidence that these are not merely different visible error types. In the tested multilingual LLMs, they often correspond to separable internal stages. ...

TL;DR for operators Choosing between a and an depends on how the next word sounds, even when spelling misleads: a university but an hour. Kim and Lee find that a single sound-related direction learned from ordinary English cases generalizes to these spelling-sound exceptions, reaching 100.0% accuracy for Llama, 95.1% for Qwen, and 98.0% for Gemma.1 More importantly, this feature is not merely decodable. When researchers hold a synthetic nonce embedding fixed and change only its position along that direction, the models shift between preferring a and an. ...