Checklist Capital: Reinforcing Agents Without Verifiable Rewards

Checklist.

It is not the most glamorous word in artificial intelligence. It does not sound like a new reasoning architecture, a sovereign model, or a mildly terrifying demo video. It sounds like something an operations manager would use before approving a vendor payment.

That is exactly why it matters.

Most enterprise agents fail to fit the clean reward structure that reinforcement learning likes. A coding benchmark can verify whether tests pass. A math problem can verify the final answer. A database query can sometimes verify whether a returned value matches the expected record. But business agents live in a less cooperative universe. They ask clarification questions, call internal tools, respect constraints, recover from missing information, and produce replies that are useful without being exactly predictable.

The annoying part is not that these behaviors are subjective. The annoying part is that they are only partly subjective. A procurement agent either checked the budget constraint or it did not. A customer-service agent either used the latest account record or it did not. A finance assistant either disclosed the assumption behind a forecast or it did not. The whole answer may be open-ended, but many of its obligations are observable.

That is the opening exploited by CM2, short for Checklist Reward for Multi-turn Multi-step Agentic Tool Use. The paper proposes a reinforcement learning framework that replaces verifiable outcome rewards with evidence-grounded binary checklist rewards for tool-using agents operating across multiple dialogue turns and multiple tool-use steps.¹

The important move is not simply “use rubrics.” Rubrics have been circling AI evaluation for a while, usually with the enthusiasm of people who have discovered that vague scalar scores are, in fact, vague. CM2’s sharper contribution is the separation of two questions that are too often merged:

1. What should be evaluated?
2. Where should the reward be assigned during training?

Its answer is the paper’s central design rule: dense criteria, sparse assignment.

In plainer business language: judge many concrete obligations, but do not inject noisy rewards at every tiny step just because the system technically allows it. Dense feedback sounds scientific. In a noisy agent environment, it may just be a faster way to teach the model nonsense with confidence.

CM2 separates what to judge from where to assign credit

CM2 begins with a practical observation. Multi-turn tool agents generate trajectories, not single outputs. A single user request may involve a chain like this:

User query
→ agent reasoning
→ tool call
→ tool response
→ more reasoning
→ another tool call
→ final reply
→ next user turn

In a real workflow, the final reply is only one visible artifact. The agent may have already succeeded or failed several times before the user sees anything. A final scalar reward compresses all of that into one number. The compression is convenient. It is also where useful diagnostic information goes to die.

CM2 instead labels each turn with a checklist. Each checklist item is a binary question, grounded in evidence from the trajectory. The item includes a focus area, a pass condition, failure examples, dependency links, a strictness flag, and a weight. In the paper’s example, an item may ask whether the assistant proposed cheaper alternatives instead of generating promotional content when the user’s budget constraint was violated.

This is the first useful mental model for business readers:

This turns open-ended judging into something closer to classification. The judge does not need to answer, “How good was the agent?” It answers smaller questions: Did the agent call the right tool? Did it use the returned value? Did it avoid continuing after a failed prerequisite? Did it answer the actual user request rather than performing the usual ritual of sounding helpful while drifting sideways?

That last behavior is common enough that it should probably have its own KPI.

The mechanism: checklist rewards make agent behavior decomposable

CM2’s reward is built around whether checklist items become newly satisfied during a trajectory.

For a dialogue turn, the system tracks whether each checklist item is satisfied before and after a step. A reward is assigned when an item flips from unsatisfied to satisfied, provided its dependencies have already been met. Conceptually:

The exact notation matters less than the operational idea: the reward is not “the final answer felt good.” It is “this step caused a specific required behavior to become true.”

The paper also introduces backfilling for the step-level version. If an item requires several steps to satisfy, the framework can attribute credit backward to earlier eligible steps. This is meant to address a real credit-assignment problem: in long tool workflows, the step that sets up success may not be the step where the checklist item finally becomes visibly satisfied.

But CM2 does not stop at designing a detailed reward. It then asks whether detailed reward assignment actually helps.

That is where the paper becomes more interesting than the usual “we made a better reward model” story.

The easy mistake is to make reward assignment too dense

CM2 defines two different kinds of granularity.

The natural engineering instinct is to make both fine-grained. More signal should mean better training. More measurements should mean better control. More dashboards should mean better management, which is why many organizations now own dashboards that no one reads.

CM2’s experiments push against that instinct.

The paper compares trajectory-level, turn-level, and step-level advantage estimation. In the validation reward curves, finer assignment improves faster early in training. Step-level rewards move quickly. Turn-level rewards also improve faster than trajectory-level rewards at first. Then the finer-grained variants become unstable. Step-level assignment shows the sharpest collapse. Turn-level assignment also degrades. Trajectory-level assignment improves more slowly but remains more stable.

The paper’s interpretation is noise amplification. Checklist rewards reduce the randomness of open-ended LLM judging by turning evaluation into binary, evidence-grounded decisions. They do not remove noise completely. Tool responses may be simulated. LLM judges may vary. Group-relative normalization can magnify small reward differences into misleading gradient updates. When reward is assigned at every small step, the residual noise enters the optimization process more often.

This is the misconception the paper quietly punctures: denser reward assignment is not automatically better.

The replacement belief is more useful: dense criteria are valuable because they specify what good behavior means; sparse assignment is valuable because it prevents the training loop from overreacting to noisy micro-judgments.

That distinction is easy to miss, and expensive to learn by accident.

The training pipeline is mostly quality control before RL

CM2’s pipeline has six major stages:

1. Filter synthetic tool-use trajectories.
2. Compress chain-of-thought content to reduce context length.
3. Train a cold-start supervised fine-tuned model.
4. Label per-turn checklists post hoc.
5. Run rollouts in a simulated tool environment.
6. Optimize with GRPO using checklist rewards.

The dataset begins from the tool-calling subset of NVIDIA’s Nemotron post-training dataset, containing 310,000 synthetic tool-use dialogues. Rule-based filtering reduces it to 280,000. LLM-based filtering reduces it much further, to 30,000 high-quality samples. From there, the authors sample 8,000 examples for cold-start SFT and retain another 8,000 complex multi-turn, multi-step dialogues for RL, with 500 held out for validation.

That filtering ratio deserves attention. The paper is not saying, “Synthetic data is magically enough.” It is saying synthetic data becomes useful after aggressive quality control. The difference is not cosmetic. Bad synthetic trajectories are not harmless. They are training instructions with the confidence of machine-generated paperwork.

The checklist labeling is also post hoc. The system does not ask humans to manually construct every reward. An LLM labels each turn by inferring the intended behavior and decomposing it into binary observable items. The authors report an average cost of roughly $0.10 per trajectory for checklist annotation.

For many organizations, that number is not the main cost. The main cost is agreeing internally on what the checklist should mean. The paper automates labeling from trajectories, but business adoption still requires process owners to define which failures are tolerable, which are critical, and which dependencies reflect real operational logic.

In other words: the model can help write the checklist. It cannot decide whether your compliance department is serious. That remains a human governance problem, tragically immune to GPU acceleration.

Simulated tools reduce engineering cost, but they also define the boundary

CM2 trains in an LLM-simulated tool environment containing more than 5,000 tools. The simulator uses a hybrid mechanism. If a generated tool call exactly matches a recorded tool name and arguments, the system replays the recorded response. If not, an LLM simulates the tool response using in-dialogue tool input-output examples as few-shot context.

This is a serious practical contribution. Building executable environments for thousands of tools is expensive. Maintaining them is worse. Anyone who has connected production APIs to an agent loop knows the tool environment becomes a second product: schemas change, permissions fail, returned fields drift, rate limits appear, and suddenly your “agent research” is a middleware maintenance business with a fancy README.

CM2 avoids that bottleneck by simulating tool responses during RL. That makes broad tool coverage possible without engineering every API.

But this is also a boundary. Simulated tools are useful for scalable training and stress-testing behavior patterns. They are not a substitute for production integration. A simulator can teach an agent the shape of a workflow. It may not expose the awkward failures that real systems produce: stale permissions, partial data, mismatched schemas, corrupted records, policy exceptions, or the ancient enterprise curse known as “the field exists but means something different in this region.”

The business implication is clear: simulated tool environments are excellent for pre-deployment training and evaluation. They should not be treated as proof that the same agent is production-safe.

The experiments support a recipe, not a universal law

The paper’s evidence has several layers. They should not be read as one undifferentiated “CM2 wins” paragraph.

Starting from an 8B base model, CM2 improves the cold-start/SFT trajectory in three benchmark families.

On τ²-Bench, the cold-start SFT model scores 18.59 average accuracy, while RL on the complex multi-turn/multi-step dataset reaches 26.76. That is a gain of just over eight points. The paper also notes an important mismatch: CM2’s RL training uses a maximum context length of 10k and up to 30 turns, while τ²-Bench can require more than 30k context and up to 200 turns. Under that mismatch, the CM2 model lags some open-source baselines. With an additional 5,000 in-domain synthetic examples, a CM2 variant reaches 41.39 average accuracy and surpasses the listed open-source baselines.

This is not a minor caveat. It means the result is strong, but context length and domain match matter. Long-horizon agents do not become magically robust because the reward is better. They still need training conditions that resemble the deployment horizon.

On BFCL-V4, CM2 reaches 36.50 overall accuracy on the Multi-Turn subset, compared with 26.75 for further SFT on the RL dataset and 19.37 for cold-start SFT. On the Web Search subset, CM2 reaches 27.50 overall, compared with 13.50 for further SFT and 14.00 for cold-start SFT. The pattern is not subtle: RL with checklist rewards is doing more than memorizing synthetic trajectories.

On ToolSandbox, CM2 reaches an overall score of 68.20, compared with 56.19 for cold-start SFT and 55.32 for further SFT on the RL dataset. It also exceeds the listed 30B-A3B and 8B-Thinking open-source baselines on overall score.

The tempting sentence would be: “CM2 beats larger models.” The more precise sentence is: \ast\astunder these benchmark settings, an 8B-base-derived model trained with CM2 can match or exceed several similarly relevant open-source baselines, including the model used for judging in parts of the pipeline.\ast\ast

That is still impressive. It is just less likely to embarrass itself in front of someone who read the table.

The central business value is diagnosis before training

The immediate business use of CM2 is not necessarily full RL training. Most companies will not casually spend 64 GPUs for 680 hours to improve an internal agent. Some will. Most should first steal the evaluation idea.

Checklist rewards create a bridge between policy, process, and model behavior. A customer-service workflow can be decomposed into criteria such as:

This creates value even before RL. It can be used for evaluation, regression testing, audit sampling, agent comparison, and failure analysis. Training is the later stage. Diagnosis is the first product.

That ordering matters. Many enterprise AI programs jump from “the agent demo works” to “how do we automate the workflow?” CM2 suggests a more disciplined sequence:

Define obligations
→ Convert obligations into checklist items
→ Evaluate agent trajectories
→ Identify recurring failure modes
→ Improve prompts, tools, data, or model behavior
→ Consider RL or fine-tuning only where evaluation shows persistent gaps

The checklist is not just a reward. It is an operational specification. It makes vague quality expectations inspectable.

This is especially relevant for workflows where final outcomes are delayed or hard to verify. A legal research assistant may not have an instant correctness label. A procurement assistant may not know whether a vendor recommendation was ultimately optimal. A finance assistant may produce a scenario analysis whose value depends on later market conditions. In such cases, exact outcome rewards are weak or unavailable, but process obligations remain observable.

CM2’s deeper business lesson is that agent quality can be managed through structured behavioral evidence rather than final-answer worship.

What Cognaptus infers for deployment

The paper directly shows that checklist rewards can improve multi-turn, multi-step tool-use agents over SFT baselines in benchmark settings, especially when reward assignment is kept coarse enough to reduce noise amplification.

For business use, Cognaptus would infer three practical pathways.

The highest-confidence application is evaluation. It requires the least infrastructure and gives immediate information. A company can start by collecting real or simulated agent trajectories, generating checklist items, and reviewing which obligations fail most often.

The medium-confidence application is offline improvement. If a workflow has enough repeated structure, checklist failures can guide prompt changes, retrieval fixes, tool-schema redesign, supervised fine-tuning, or targeted RL.

The lowest-confidence application is direct production assurance. CM2 is not a certification method. Passing checklist rewards in a simulated environment does not prove an agent is safe under live business conditions. It proves the agent has learned behavior patterns that satisfy the specified criteria under the tested conditions.

That is useful. It is not magic. We should be grateful for the distinction.

The economics are attractive, but not free

The paper reports checklist labeling at about $0.10 per trajectory, which sounds cheap. For an enterprise team, that figure should be read correctly.

The labeling cost may be low. The governance cost is elsewhere:

• selecting representative trajectories;
• defining critical versus non-critical failures;
• validating generated checklist items;
• mapping checklist dependencies to real workflow rules;
• choosing acceptable false positive and false negative rates;
• integrating evaluation with deployment monitoring;
• deciding when failures trigger retraining, escalation, or rollback.

The training cost is also nontrivial. CM2’s RL run uses substantial compute. That does not weaken the paper; it clarifies the adoption path. Most organizations should not begin with full CM2-style RL. They should begin with checklist evaluation and only escalate to training when the return justifies it.

A practical adoption ladder would look like this:

This ladder is less exciting than announcing “autonomous enterprise agents.” It also has a better chance of surviving contact with procurement, legal, and IT security.

Where the paper stops

The paper is careful enough to expose its own boundaries.

First, the training data is synthetic and aggressively filtered. That is reasonable for scale, but it means the resulting policy may reflect the distribution and blind spots of synthetic trajectories.

Second, checklist labeling and filtering depend on powerful LLMs. The framework reduces open-ended reward ambiguity, but it still relies on LLM judgment. Binary labels are more stable than scalar scores; they are not automatically true.

Third, the tool environment is simulated. Hybrid replay plus LLM simulation is a clever solution to tool-infrastructure cost, but production tools are less polite than simulated ones. Real systems fail in ways that are boring, inconsistent, and commercially significant.

Fourth, the benchmark results show strong improvements, but not uniform superiority under all conditions. The τ²-Bench context mismatch is a useful warning: long context and very long turn sequences remain hard.

Fifth, the cost profile matters. The framework uses 8B models, a 30B-A3B model for simulation and judging, GPT-5 for filtering and labeling, group sizes up to 48, and large-scale GPU training. The method is scalable in the research sense. In business, scalability also means budget approval.

None of these limitations make the paper weak. They make the implementation path concrete.

The reward is the product requirement in disguise

CM2 is valuable because it shifts attention from final answers to enforceable behavior.

For enterprise agents, this is the right direction. The question is rarely “Can the model produce a plausible response?” That bar was cleared some time ago, often too enthusiastically. The harder question is whether the agent follows the obligations of a workflow across turns, tools, constraints, and user-facing communication.

Checklist rewards give those obligations a machine-usable form.

The paper’s most useful message is not that checklists are better than all reward models, or that simulated tools solve production deployment, or that step-level rewards are forever doomed. The more durable message is narrower and stronger:

When exact verification is unavailable, decompose open-ended behavior into evidence-grounded binary obligations. Then be careful where you inject the reward signal, because noisy detail is still noise.

That is a serious lesson for anyone building AI agents inside real organizations.

Not every AI breakthrough looks like a new model. Some look like a better way to write down what “good work” means before asking a machine to optimize it.

In enterprise AI, that may be the more radical act.

Cognaptus: Automate the Present, Incubate the Future.