Model Interpretability

Opening — Why this matters now Reasoning models are having a moment. Latent-space architectures promise to outgrow chain-of-thought without leaking tokens or ballooning costs. Benchmarks seem to agree. Some of these systems crack puzzles that leave large language models flat at zero. And yet, something feels off. This paper dissects a flagship example—the Hierarchical Reasoning Model (HRM)—and finds that its strongest results rest on a fragile foundation. The model often succeeds not by steadily reasoning, but by stumbling into the right answer and staying there. When it stumbles into the wrong one, it can stay there too. ...

Imagine debugging a black box. Now imagine that black box occasionally narrates its thoughts aloud. That’s the opportunity—and the fragility—presented by Chain-of-Thought (CoT) monitoring, a newly emergent safety paradigm for large language models (LLMs). In their recent landmark paper, Korbak et al. argue that reasoning traces generated by LLMs—especially those trained for explicit multi-step planning—offer a fleeting yet powerful handle on model alignment. But this visibility, they warn, is contingent, brittle, and already under threat. ...