LLM Interpretability

Audit is easy when the answer is a single lookup. A customer asks, “What is your refund policy?” The model quotes the policy paragraph. We check whether the quoted paragraph came from the right source. Very civilized. Everyone goes home early. But real enterprise LLM work is rarely that tidy. A compliance assistant reads a contract, extracts obligations, compares them with internal policy, reasons through exceptions, and writes a recommendation. A research assistant reads multiple sources, builds an intermediate summary, then answers a question from that summary. A support agent reads a user history, infers the likely issue, then proposes the next action. In these cases, the final sentence may depend on prompt evidence and on earlier generated text. ...

TL;DR for operators This paper is not mainly about whether an LLM can forecast stock moves from news. That storyline is already crowded, noisy, and full of people discovering that backtests look unusually handsome when nobody has yet met execution costs. The more useful contribution is different: it shows a way to inspect and adjust the internal concepts an LLM activates while processing financial text. ...

TL;DR for operators The paper’s useful claim is not simply that some chain-of-thought sentences matter more than others. That would be true, mildly interesting, and about as operationally helpful as saying some meetings should have been emails. The sharper claim is that the sentences that steer reasoning are often not the visible calculations. They are planning moves, re-checks, uncertainty statements, and backtracking moments: the places where the model chooses a route, notices a contradiction, or decides to verify a previous result. Bogdan, Macar, Nanda, and Conmy call these pivotal sentences thought anchors.1 ...