Fix the First Bad Token, Not the Whole Answer

TL;DR for operators

When a model answer is mostly right but fails at one point, rewriting the entire response may waste both human effort and information about what the model already does well. onPanda turns that situation into a different annotation workflow: find the first inappropriate token, replace it, then let the same model continue from the corrected prefix.

In the paper’s controlled comparison, this reduced median annotation time from 681 seconds with manual post-editing to 330 seconds. The resulting responses also stayed much closer to the model’s own generation distribution: perplexity increased only 0.86% relative to a resampling baseline, versus 36.31% for manual post-editing.

The more consequential contribution is structural. One correction interaction can yield a qualified SFT example, preference pairs, and a record of exactly where the original trajectory went wrong and what replaced it. For teams operating models that are already close to acceptable behavior, this could increase the supervision recovered from each minute of expert review.

The boundary is equally important. The paper evaluates annotation efficiency, data properties, deployments, and correction capability. It does not show that training on these token-level correction signals produces a better downstream model. The workflow also depends on errors remaining sparse and on inference infrastructure that supports prefix continuation and token-probability access.

The expensive part may be everything after the first mistake

Suppose a model produces a 500-token answer and the first 300 tokens are acceptable. At token 301, it makes a substantive error. A human reviewer now has several choices: rewrite the response, choose among several complete alternatives, or repair the local failure and preserve the usable trajectory.

Those choices do not produce equivalent data. Whole-response preference ranking requires relatively little writing, but its feedback is coarse: one response wins, another loses. Manual post-editing can produce a qualified answer, but extensive rewriting moves the final text away from what the current model would naturally generate.

onPanda, introduced by Yang and colleagues,¹ takes the third route. The annotator identifies the earliest inappropriate token, replaces it either with a probability-ranked candidate or free-form text, discards the invalid suffix, and lets the same model regenerate from the corrected prefix.

This turns annotation into trajectory repair. The model keeps responsibility for most of the response; the human intervenes where the trajectory first becomes unacceptable.

Sparse repair changes what one annotation session produces

The interface is only part of the contribution. The annotation tree records the rejected token or span, its replacement, the correction position, and the trajectories before and after the intervention.

That provenance allows one workflow to generate several distinct training artifacts.

A qualified final trajectory can become SFT data. Ancestor-descendant versions can form preference pairs. The correction point itself supplies position-specific positive-negative supervision: at this position, under this prefix, one continuation was rejected and another was chosen.

In the controlled study, onPanda reached 100% SFT coverage across the 21 prompts and produced 7.43 preference pairs per prompt. Argilla’s four-candidate ranking workflow produced six preference pairs per prompt but yielded a qualified response for only 11 of 21 prompts. POTATO post-editing reached 100% SFT coverage but produced only 0.95 preference pairs per prompt.

The operational distinction is therefore larger than “faster editing.” The workflow changes the amount and granularity of supervision extracted from the same human review event.

The controlled evidence supports the mechanism, within a narrow setting

The paper compares onPanda with POTATO manual post-editing and Argilla four-candidate ranking using three trained annotators and 21 image-description prompts. A Latin-square design rotates workflows across prompt groups, with a common Qwen3.5-35B-A3B rollout model and shared sampling settings.

The time result is striking but should be read narrowly. onPanda’s 330-second median is 51.5% below POTATO’s 681 seconds, yet it is almost identical to Argilla’s 336 seconds. The gain is therefore not simply “token correction makes annotation twice as fast.” Relative to ranking, the stronger result is that similar median annotation time produces qualified responses for every prompt while preserving richer correction provenance.

The perplexity comparison supports the paper’s distributional argument. If most of the final trajectory is still generated by the same model, the result should resemble that model’s sampling distribution more closely than extensively human-rewritten text. onPanda’s perplexity was 1.181, against a 1.171 resampling baseline; POTATO reached 1.596.

Production statistics are consistent with this sparse-intervention mechanism. Across the reported deployments, 97.0% of tokens in qualified responses were model-generated, 2.1% were selected from model candidates, and 0.9% were manually typed.

These numbers make the mechanism plausible at operational scale. They do not isolate the causal contribution of the interface from the annotation paradigm, because the controlled comparison changes both.

Agent review can happen before the tool call becomes irreversible

The same interaction model extends beyond plain text. onPanda renders structured model outputs so annotators can correct reasoning, ordinary content, or tool-call arguments, then parse the corrected stream back into structured messages. With connected environments, a corrected action can be executed and the returned tool result fed back into the continuing trajectory.

For agent developers, that creates a concrete intervention point: human review can sit immediately before a consequential action rather than after a completed trajectory.

The production deployment includes 1,257 agentic sessions, with a median annotation time of 31.31 minutes and an average 5.23 tool calls per session. This demonstrates that the architecture can operate over long, structured interactions.

It does not establish an efficiency advantage for agent annotation. The paper reports no controlled agent-workflow comparison, so the agentic evidence supports system capability and deployment feasibility rather than a measured productivity gain.

The benchmark exposes a capability gap behind the interface

A human annotation system based on first-error correction raises a natural automation question: can current models perform that inspection themselves?

Panda-CVL turns the task into a benchmark. Models must first determine whether a response is already acceptable. If not, they must locate the first inappropriate token and supply the reference correction.

The reported results separate formatting competence from diagnostic competence. Seven of nine reasoning models exceed 90% format compliance, yet correction accuracy on not-good responses remains below 16% for every evaluated model. The best overall F1 is 17.09%.

That gap matters operationally. Generating a syntactically valid correction record is not the same as reliably identifying the first substantive failure. The benchmark therefore supports keeping a human in the loop today more strongly than it supports automating the annotation process.

Where the operating case breaks down

Cognaptus’s interpretation is that token-level correction is best matched to near-capable policies with sparse failures.

If a model is already mostly correct, preserving its valid trajectory can reduce unnecessary rewriting while producing unusually rich supervision. That makes the approach relevant to expert review settings where the cost is not generating an answer from scratch but repeatedly repairing localized defects.

Three conditions weaken that case.

First, dense errors remove the efficiency advantage. If annotators must intervene repeatedly, both labor savings and fidelity to the original model distribution deteriorate.

Second, the system depends on specific inference capabilities, including continuation from an assistant-message prefix and top-$k$ token probabilities. Probability refresh requires prompt log probabilities, and structured reasoning or tool-call support can require model-specific response templates.

Third, “on-policy” is temporary. Fidelity is measured relative to the rollout model used during annotation. Once the policy changes enough, previously collected corrections may no longer closely represent what the new model would generate.

That suggests a recurring-data interpretation rather than a permanent-data interpretation: the strongest case is for collecting corrections against the policy currently being operated.

Better annotation is demonstrated; better post-training is not

The paper makes a persuasive systems case for treating localized human correction as a first-class data-generation workflow. It shows lower annotation time than manual post-editing in its controlled setting, close-to-baseline rollout-model perplexity, low reported workload, production use across multiple modalities, and a mechanism for extracting several supervision types from one interaction.

The unresolved step is downstream learning.

No controlled experiment trains a model on the resulting token-level correction signals and measures whether that model improves. The paper’s proposed correction-oriented post-training directions remain future work.

That distinction matters because onPanda has already demonstrated something valuable without resolving that question: when a capable model makes sparse mistakes, the human annotation unit does not have to be the whole response. It can be the point where the trajectory first goes wrong.

Cognaptus: Automate the Present, Incubate the Future.