Catch the Drift Before the Answer: Reasoning Trajectories as a Runtime Control Surface

TL;DR for operators

When a model is midway through a difficult task, the expensive choice is often whether to leave it alone, verify it, or spend more inference on correction. Doing that for every request wastes compute and can damage answers that were already on track.

Lihao Sun and colleagues show that the model’s internal state changes systematically as reasoning progresses, and that late-stage changes contain a useful warning signal for eventual failure.¹ Their trajectory-based correctness features reach a seed-averaged best-layer ROC-AUC of about 0.85, versus 0.765 for the strongest reported LogitLens baseline and 0.649 for reasoning step count alone.

The more consequential result is what happens after detection. Unconditional inference-time interventions sometimes reduce accuracy sharply. Gating those interventions with the correctness predictor restricts changes to roughly 12% of examples and avoids much of that collateral damage. A separate trajectory-steering method improves six- and seven-step GSM8K accuracy by about 7.6–7.7 percentage points, while preserving at least 97% of originally correct solutions in those settings.

For production systems, this suggests a runtime control pattern: monitor the reasoning process, escalate only trajectories that appear to be drifting, and treat extra computation as a conditional resource rather than a default. The boundary is substantial: the evidence comes mainly from one 8B model family and a small benchmark set, and an error signal is easier to obtain than a reliable repair.

The failure signal appears before the answer does

Suppose a model is working through a multi-step problem. Before its final answer appears, a serving system has limited evidence about whether intervention is warranted. Output confidence is not yet available in its final form. Reasoning length is observable, but a long answer can reflect either genuine difficulty or wasted computation.

The paper’s central finding is that the internal process itself contains more information.

At each reasoning-step boundary, the authors extract the model’s hidden state. Those states occupy regions that are sufficiently distinct for simple linear classifiers to identify which reasoning step the model has reached. The separation becomes stronger for later steps at deeper layers. Step 1 is classified above 0.99 accuracy at every layer, while Step 2 reaches that level by layer 2.

This is more than a visualization result. Comparable structure appears in Llama 3.1 8B Base, Instruct, and DeepSeek-R1-Distill-Llama-8B. Cross-model probes also transfer strongly. That makes reasoning post-training look less like the creation of a new step structure and more like a reorganization of where that structure becomes accessible, particularly around termination.

The authors also test the obvious formatting objection. Under minimal freeform prompting, the Instruct model uses explicit Step X: formatting in 64.5% of GSM8K responses, but fixed-format probes continue to identify progress in paragraphs, numbered lists, newline-separated reasoning, and single-block answers. On freeform GSM8K, best-layer probe accuracy ranges from 0.83 to 0.88 for Steps 2–5.

That test is best read as a robustness check: it weakens the explanation that the probe is merely decoding visible step tokens.

Once successive internal states can be compared, the paper describes reasoning as a representation trajectory: a sequence of model states moving through this step-indexed space.

Correct and incorrect paths separate late

The trajectories of correct and incorrect solutions look similar early in reasoning. The separation emerges later, with the largest reported distance differences occurring around the final reasoning step and the move toward the answer.

That timing matters. If failure were identifiable immediately, runtime monitoring could make a decision after the first few tokens. The evidence instead suggests that some errors become internally distinguishable only after enough reasoning has accumulated.

Using late-step trajectory features, the authors obtain substantially stronger correctness prediction than from early-step geometry. Peak ROC-AUC reaches 0.87. Averaged across seeds at the best layer, trajectory features score 0.852 ± 0.039.

This comparison is main evidence, not an auxiliary ablation: it establishes that the trajectory contains information beyond two more obvious runtime proxies.

The authors separately address a likely misreading. Longer chains are often harder and more error-prone, so perhaps the classifier is simply learning length. Under length-balanced resampling, trajectory features still reach 0.847 ± 0.006 AUC. That robustness test does not prove independence from every difficulty variable, but it makes raw chain length an inadequate explanation of the reported signal.

Detection changes the economics of intervention

A warning signal becomes operationally interesting only if it changes what the system should do next.

The paper compares always-on inference interventions with interventions applied only when the correctness predictor flags an example. The contrast is sharp for several token-based interventions.

For example, adding “Check” unconditionally reduces GSM8K accuracy by 11.70 percentage points; predictor gating changes that result to +0.23 points. “Wait” moves from −30.50 points unconditionally to −0.68 points when gated. Even the milder “Step” intervention moves from −1.59 to +0.91 points.

The predictor triggers on only about 12% of examples.

The result does not show that gating turns every intervention into a strong correction mechanism. Net gains remain modest because some flagged failures cannot be repaired, and false positives can still perturb correct reasoning. What it does show is that where an intervention is applied can matter more than whether the intervention exists at all.

Cognaptus inference: this is the clearest route to production value. A runtime supervisor could use an internal trajectory score to decide which requests receive an additional verifier, retry, alternative reasoning pass, or more expensive model call. The affected user is the system operator allocating inference budget; the decision is whether to escalate a request; the relevant condition is evidence of late-stage trajectory drift; the boundary is that the predictor requires calibration for the deployed model and task.

The same geometry can also steer the reasoning process

The authors go one step further by defining a reference path from correct training examples. At reasoning step $j$, the current projected state $z_j$ is compared with the mean correct state $\mu_j$:

Here, $\delta_j$ measures local departure from the reference path, while $D_j$ tracks accumulated departure. When deviation passes a threshold, the method applies a low-rank correction toward the step-specific reference.

This intervention is most effective on longer GSM8K chains. Six-step problems improve from 75.44% to 83.04%, and seven-step problems from 67.69% to 75.38%. Reported preservation of originally correct solutions remains at or above 97%.

Those results are stronger evidence for targeted correction than the earlier token-injection experiments, but they remain conditional on an important assumption: correct examples must define a meaningful reference trajectory. Tasks with several qualitatively different valid solution paths may not satisfy that assumption as cleanly.

The paper also identifies a direction associated with approaching the final-answer state. Adding that direction tends to shorten reasoning; moving against it prolongs reasoning. At moderate strengths, reasoning length changes approximately monotonically.

This is a control result, not evidence that longer reasoning is intrinsically better. Strong steering away from termination can produce repetitive loops. The experiment therefore separates controlling inference length from creating useful additional computation.

What transfers, and what does not

The structural step geometry transfers better than the correctness signal.

Probes trained on fixed-form GSM8K remain strong on MATH-500 and MMLU: best-layer accuracies for Steps 1–3 are 0.98–1.00, with later steps still largely separable. That supports the interpretation that broad reasoning progress may have some task-invariant structure within the evaluated models.

Correctness prediction is more task-sensitive. Its out-of-distribution performance drops on MATH-500, MMLU, and freeform GSM8K. For deployment, that distinction is consequential. A generic progress monitor may transfer reasonably well while the part that decides “this trajectory is probably wrong” still requires task-specific calibration.

The broader external-validity boundary is narrower still. The principal experiments use three training regimes built on the Llama 3.1 8B family and evaluate GSM8K, MATH-500, and MMLU. The paper does not establish the same geometry for other model families, larger scales, open-ended research tasks, program synthesis, or workflows where multiple reasoning paths can all be correct.

Runtime reasoning may need observability before it needs more tokens

The paper changes the framing of test-time reasoning in a practical way. If a system can observe how reasoning evolves internally, inference-time compute no longer has to be allocated only from prompt difficulty, elapsed tokens, or final confidence.

The stronger design is conditional: monitor the trajectory, wait until the signal becomes informative, then decide whether to verify, retry, steer, continue, or stop.

The experiments do not yet make that a general deployment recipe. Correctness signals are less portable than step structure, repairs remain imperfect, and aggressive length manipulation can turn additional computation into repetition. But within the evaluated setting, the paper supplies a concrete reason to treat reasoning as something a serving system can supervise while it is happening rather than merely score after it ends.

Cognaptus: Automate the Present, Incubate the Future.