The Verifier Already Knows: Turn Pass/Fail Checks Into Training Credit

TL;DR for operators

A long-running agent may execute dozens of actions before receiving one final pass/fail result. Applying that same terminal signal across the whole trajectory leaves training with little information about which earlier actions actually contributed to satisfying the task.

VICT¹ treats an existing programmatic verifier as a source of selective training credit. It decomposes the verifier into explicit checks, determines which checks are relevant to the rollout-level preference, and assigns additional credit only when trajectory evidence links a particular action to one of those checks. When the required evidence is missing or the verifier reconstruction is unreliable, the method falls back to the original outcome-based advantage.

The reported gains are substantial within the paper’s primary setting. With Qwen2.5-7B, VICT reaches 93.7% average success on ALFWorld and 83.6% strict success on WebShop, compared with 77.6% and 66.1% for outcome-only GRPO under the matched training protocol. Ablations also show that simply making rewards denser, rewarding the final commit, or assigning credit to the temporally nearest action does not reproduce the full result.

For teams training agents against deterministic workflow checks, database constraints, or transaction rules, the paper suggests a concrete design choice: those checks can potentially become training-time attribution infrastructure rather than remaining final evaluators. The cost is extra verifier engineering, maintenance, and 11.9–16.7% reported training-time overhead. The evidence does not establish the same benefit for holistic learned judges or verifiers whose internal evidence cannot be observed.

A final score hides information the verifier already computed

Consider an agent that searches, edits records, calls tools, and commits a transaction. The workflow ends with a verifier deciding whether the task succeeded.

Conventional outcome-only reinforcement learning can reduce everything the verifier checked to a single scalar. One rollout receives a positive group-relative advantage, another a negative one, and that rollout-level signal becomes the anchor for every action taken along the way.

The information loss is easy to overlook. A programmatic verifier may already know that the correct product attributes were selected, a required field was written, a constraint remained satisfied, and the final commit matched the requested state. Those are different facts tied to different parts of the trajectory. Collapsing them into one score removes the structure that could help training distinguish responsible actions from merely adjacent ones.

VICT asks whether that structure can be preserved long enough to guide the policy update.

VICT makes credit conditional on evidence, not just a satisfied sub-check

The method first exposes the terminal verifier as a collection of smaller executable or evidence-backed checks, together with their dependencies, score aggregation, evidence bindings, and commit conditions. The paper calls these checks verifier atoms.

This decomposition is not trusted automatically. The instrumented representation must reconstruct the original verifier within a fixed tolerance on held-out trajectories and verifier-relevant mutations. Across the four reported domains, reconstruction is 99.7–100.0% and mutation conformance is 98.6–99.6%.

The next step is deliberately selective.

For each rollout, VICT uses the direction of the original group-relative advantage to search for a compact, dependency-closed subset of verifier checks whose counterfactual modification would change the verifier score in the corresponding direction. This is the method’s dependency-valid core: not every satisfied check, but a logically consistent subset relevant to why the rollout is preferred or disfavored.

Even core membership is insufficient to credit an action. VICT also requires an observable proof edge linking an action to a check through fixed witnesses such as a write, reveal, commit, or violation recorded in the trajectory.

The verifier-derived action correction is therefore restricted to supported core checks:

The original rollout-level advantage remains in place. VICT adds only a scaled and clipped correction:

If conformance fails, the relevant marginal is unavailable, or no supported proof edge exists, the method abstains from adding the correction. Reported abstention ranges from 8.1% on ALFWorld to 15.8% on tau-bench Airline.

That distinction separates VICT from simply turning every verifier sub-check into a dense reward.

The ablations test whether attribution structure is doing real work

The main comparison establishes that verifier-backed credit can outperform outcome-only training in the reported environments. The ablations address a different question: whether VICT’s extra structure is necessary, or whether almost any finer-grained reward would work.

With Qwen2.5-7B, every tested simplification underperforms full VICT:

These are three-seed results. Because the components interact, the paper explicitly does not treat the gaps between individual variants as statistically independent causal effects.

The pattern is nevertheless informative. Dense atom rewards improve substantially over outcome-only GRPO but remain below full VICT. Temporal proximity is also insufficient. Most strikingly, randomizing the action-to-check links largely collapses the gains.

The evidence therefore supports a narrower mechanism claim: making reward information finer helps, but verifier relevance and the alignment between a particular check and observable trajectory evidence appear to matter beyond reward density alone.

Proof-supported credit is auditable attribution, not causal proof

The method’s inspectability can invite a stronger interpretation than the paper supports.

A proof edge says that an observable action satisfies VICT’s verifier-defined conditions for receiving credit. It does not establish that the action was causally necessary for success, that no alternative action would have worked, or that the credited action was optimal.

This boundary is central to how the diagnostics should be read. VICT reports 84.9–92.4% proof coverage across the four domains, while the eligibility invariant passes at 100%. The latter is an implementation property: whenever the system emits a non-zero correction, the specified eligibility conditions have been satisfied. It is not evidence that the attribution recovered the unique causal explanation of the outcome.

For model-risk review, that weaker guarantee can still be valuable. A team can inspect why training credit was allowed to flow to an action and what verifier evidence supported it. That is a more concrete audit trail than an opaque learned critic score, without pretending that the trace solves causal attribution.

Existing workflow rules could become training infrastructure

The business inference is strongest where success is already defined programmatically.

A commerce agent may face transaction constraints. A back-office agent may be checked against required database state. An operations workflow may already contain permission checks, field validations, reconciliation rules, or deterministic completion criteria.

Today, such rules often sit at the end of the pipeline: they decide whether the agent succeeded. VICT suggests that some of them could also inform how the agent learns, provided teams can expose the checks, maintain their dependencies, and connect them to observable trajectory events.

The attraction is not only higher benchmark performance. The credit pathway itself becomes inspectable. A non-zero training correction can be traced to a verifier condition and supporting evidence, while unsupported cases can abstain to the base training signal.

That capability is not free. The primary verifier adapters required 118–238 lines of code and an estimated 6.5–11.5 person-hours, while training overhead was 11.9–16.7%. Those figures make verifier instrumentation an engineering investment rather than an automatic by-product of having a terminal reward.

The method stops where the verifier becomes opaque

VICT is best matched to environments with explicit acceptance criteria and logs that reveal relevant writes, observations, commits, or violations.

Its applicability becomes less direct when success depends on a holistic learned judge, important verifier state is hidden, the verifier cannot be decomposed reliably, or the main difficulty is exploration rather than attributing an observed outcome. Larger or ambiguous dependency structures can also reduce proof coverage and increase abstention.

The supplemental tau-bench results extend the evidence to Retail and Airline interactions—56.6% and 45.1% pass@1 for VICT versus 51.3% and 40.0% for Fission-GRPO—but those experiments use Qwen3-8B and a different protocol from the primary ALFWorld and WebShop evaluations. They should be treated as supporting evidence, not as a directly matched continuation of the main benchmark block.

The broader design proposition is therefore specific: when a workflow already has an explicit, inspectable definition of success, collapsing that verifier to one number may discard training information that the system has already paid to compute. VICT provides evidence that preserving some of that structure—and refusing to attribute where supporting evidence is missing—can produce both stronger learning signals and a more inspectable credit trail.

Cognaptus: Automate the Present, Incubate the Future.