MatchTIR: Stop Paying Every Token the Same Salary

Payroll is a useful metaphor for agent training because it makes the absurdity obvious.

Imagine a project team where one employee finds the right database, another enters the correct query, a third repeatedly calls the wrong API, and a fourth finally writes the report. If the report is accepted, everyone receives the same bonus. If it fails, everyone receives the same blame. Very democratic. Also very stupid.

That is roughly how many reinforcement learning pipelines still supervise multi-turn tool-using agents. The final answer is correct or incorrect; sometimes the whole trajectory receives a score; then every token or turn inside that trajectory is treated as if it contributed equally. The model is not told which tool call helped, which one was redundant, which parameter was fatal, and which recovery step saved the task.

The paper behind MatchTIR attacks exactly this bottleneck.¹ Its argument is not that tool agents need more enthusiasm, more context, or another ceremonial benchmark parade. Its argument is narrower and more useful: when an agent calls tools over multiple turns, the training signal must stop paying every token the same salary.

That small change matters because tool use is where LLMs stop being chatbots and start touching business systems: APIs, databases, search services, spreadsheets, workflow engines, internal knowledge bases, CRM records, procurement systems, and whatever legacy software is still alive because one senior employee knows where the button is. In those environments, a near-miss tool call is often not “almost right.” It is simply wrong, with excellent formatting.

The old reward model cannot tell a useful retry from a noisy one

Tool-Integrated Reasoning, or TIR, means the model does not only produce text. It reasons, calls a tool, observes the response, reasons again, calls another tool, and eventually gives an answer. The paper frames this as a long-horizon, multi-turn process. That framing is important because the difficulty is not merely selecting one function. It is managing a sequence.

Traditional reward designs usually fall into two families.

Outcome rewards are sparse. They arrive late, after the agent has already made several decisions. Trajectory rewards are denser, but still blunt. They compress a heterogeneous sequence into one scalar and then distribute that signal across the path. A correct first tool call, a pointless duplicate call, and a malformed parameter can all receive the same advantage if they live in the same trajectory.

That is the misconception MatchTIR is designed to break: a correct final answer does not prove that the reasoning path was clean. Agents can stumble into the right answer. They can recover from bad calls. They can spam tools until one happens to work. If reinforcement learning rewards all of that uniformly, it does not merely tolerate bad habits. It teaches them.

The paper’s useful observation is that many tool-use settings are not completely open-ended. A tool call has structure: tool name, parameter names, parameter values, and tool output. That structure makes the intermediate steps more verifiable than free-form reasoning text. MatchTIR exploits this property directly.

MatchTIR turns tool-call grading into an assignment problem

The central mechanism is simple enough to be worth understanding carefully.

MatchTIR compares the agent’s predicted tool calls with ground-truth tool calls. Instead of asking only “Was the final answer right?”, it asks: which predicted calls correspond to which reference calls?

That is a matching problem.

For each predicted tool call and each ground-truth tool call, the method builds a similarity score using three components:

Once these pairwise scores exist, the model faces a second problem: how to convert them into rewards without allowing reward hacking.

Directly rewarding similarity scores can be dangerous. An agent could repeatedly call nearly similar tools and accumulate partial credit. Anyone who has seen a model “try again” five times with the same broken argument can appreciate the issue. The paper’s phrase is more polite; the behavior is not.

MatchTIR proposes two assignment strategies.

The first is hard credit assignment using the Hungarian, or Kuhn–Munkres, algorithm. This enforces one-to-one matching. Each ground-truth call can reward at most one predicted call. Duplicate attempts do not get to line up at the payroll window twice.

The second is soft credit assignment using Optimal Transport. This allows probabilistic alignment between predicted and ground-truth calls. It gives smoother feedback because several predicted turns may receive partial credit relative to the reference sequence.

At first glance, soft credit sounds more elegant. In practice, the paper finds the hard version usually performs better. That is not a trivial result. It says something about tool-use environments: when the wrong parameter breaks execution, partial semantic similarity can become misleading. A function call that is “close enough” for a human explanation may still be invalid for a machine.

Soft credit is generous. APIs are not.

Turn rewards become useful only when advantages stop being uniform

Rewarding individual tool calls is not the full method. MatchTIR then aggregates tool-call rewards into turn-level rewards, because a single turn may contain multiple tool calls and policy optimization usually needs a signal attached to a turn or token range.

It also keeps an outcome-level reward, based on final answer quality. This matters. A model that learns to make locally correct tool calls but fails the final task is not useful. Local execution and global success must be trained together.

The paper’s second contribution is therefore dual-level advantage estimation. It uses two views of credit:

A simplified way to read the integrated signal is:

The exact implementation is inside a GRPO-style optimization setup, but the editorial meaning is more important than the algebra. MatchTIR does not throw away the global signal. It refuses to let the global signal drown every local decision.

That design choice fits the structure of multi-turn tool use. An early correct call can make later calls possible. A wrong call can poison the next observation. A recovery call can rescue a trajectory that would otherwise fail. Credit assignment has to preserve these dependencies instead of flattening them into one mood.

The main results show a supervision gain, not just a benchmark bump

The paper evaluates MatchTIR on three benchmarks: FTRL for in-domain evaluation, and BFCL plus ToolHop for out-of-domain evaluation. It uses Qwen3-4B and Qwen3-8B backbones, compares against vanilla models, GRPO, FTRL variants, and ToolRL variants, and trains in a Zero-RL setting with 16 rollouts per query.

The main result is easy to summarize but worth not overselling.

The 4B MatchTIR-KM model reaches an overall average of 36.07. That beats most 8B baselines in the table, though not the strongest 8B FTRL-M baseline. The correct interpretation is not “small models always beat large models.” That would be the usual internet-friendly nonsense. The better interpretation is that finer supervision can recover a meaningful amount of capability that brute parameter scaling alone does not guarantee.

The KM variant also consistently beats the OT variant in the main table. This supports the paper’s argument that strict one-to-one credit assignment is valuable when exact tool execution matters. Soft alignment may be smoother, but smooth wrongness is still wrongness. Enterprise APIs rarely accept “emotionally adjacent” parameters.

The ablations explain why MatchTIR is not one trick wearing a lab coat

The ablation study is doing more than decorating the paper with another table. Its purpose is to separate three questions:

1. Does turn-level reward help compared with outcome-only reward?
2. Does combining local and global reward help?
3. Does dual-level advantage estimation matter beyond the reward design itself?

On Qwen3-8B over FTRL, outcome-only trajectory-level estimation gives an average of 34.63. Turn-level reward improves the score. Integrated reward improves it further. The full MatchTIR design reaches 39.28, higher than variants that use only turn-level or only trajectory-level advantage.

This matters because the paper’s claim is mechanistic. If only the main benchmark improved, one could suspect a benchmark-specific trick. The ablations instead support a cleaner story: supervising intermediate tool execution helps, but it works best when combined with global task success and cross-rollout comparison.

The appendix adds another useful detail. The authors compare MatchTIR with weighted intra-trajectory variants. Those variants normalize turns inside a single trajectory, then use local signals to modify the global advantage. They perform worse than MatchTIR. The paper’s explanation is plausible: within one trajectory, local variance does not capture the relative quality of alternative paths. Multi-turn agents need comparison across possible rollouts, not just introspection inside one attempt.

That is an important distinction for business readers. Many workflow diagnostics are designed like post-mortems: examine one failed run, label the bad step, move on. MatchTIR suggests that training benefits from comparing alternative trajectories for the same task. The agent should learn not only that one path failed, but that another path handled the same situation better.

The efficiency table is the quiet business result

The most business-relevant evidence is not necessarily the headline score table. It is the tool-use efficiency analysis.

For Qwen3-4B, MatchTIR reduces tool calls by about 10.2% while raising tool-call success by 12.39 percentage points. For Qwen3-8B, it reduces tool calls by about 13.8% while raising success by 8.82 percentage points.

This is not just “better accuracy.” It is a behavioral change. The trained agent becomes more selective. It spams tools less. It fails less often. It does not merely arrive at the correct answer more often; it appears to waste fewer external actions while doing so.

That distinction matters in enterprise systems because tool calls are not abstract tokens. They can mean API bills, database load, latency, audit exposure, user-facing errors, failed transactions, and messy logs that humans later have to interpret. A model that calls fewer tools with higher success is not just smarter. It is cheaper to operate and easier to govern.

Cognaptus inference: if the same principle transfers to controlled business workflows, fine-grained tool supervision could reduce the hidden cost of agent deployment. The paper directly shows benchmark-level tool-call efficiency. It does not directly measure cloud bills, employee time saved, or incident reduction. Those are business hypotheses, not paper results. Still, they are reasonable hypotheses because tool calls have operational cost in real systems.

Hard matching is a governance feature disguised as an algorithm choice

The KM-versus-OT result deserves more attention than a casual “hard beats soft.”

Hard matching says: one predicted call gets credit for one ground-truth call. If the agent repeats a similar call, the duplicate does not keep earning reward. If the parameter is wrong, the call may receive no credit. This is a harsh regime, but structured tool execution is harsh too.

Soft matching says: partial alignment deserves partial reward. That can be useful when multiple paths are genuinely valid or when gradients need smoother shaping. But the paper finds that soft credit can reward near-miss calls that remain fatal at execution time.

For enterprise governance, this difference maps to a familiar design choice.

The paper does not claim that hard matching is universally superior. It shows that in the tested structured tool-use settings, hard one-to-one assignment performs better. The broader lesson is that reward design should respect the execution environment. When the system is brittle, the reward should not be sentimental.

The complexity tests say the pain grows with longer workflows

MatchTIR’s robustness-to-complexity analysis groups queries by the number of tool invocations required. The reported trend is intuitive: gains become more pronounced on harder tasks.

This is exactly where uniform credit assignment becomes most damaging. In a one-step tool call, final success and intermediate correctness are almost the same signal. In a six-step workflow, they diverge. An early error may be corrected later. A correct early step may be buried under later noise. A redundant call may not prevent final success but still teaches inefficient behavior.

Long-horizon tasks therefore create a credit fog. MatchTIR’s turn-level rewards act like visibility markers inside that fog.

The hyperparameter analysis reinforces the same point. Increasing the penalty for unmatched or redundant calls improves Solve-P, meaning tool-use precision, but can slightly reduce Solve-R, meaning completeness. In plain English: punish bad calls too aggressively and the model becomes conservative. It avoids mistakes, but may also avoid necessary exploration.

The discount factor analysis points in the other direction. A higher discount factor improves performance across metrics because earlier turns often affect later outcomes. Multi-turn tool agents are not a pile of independent API calls. They are sequences where the cost of a bad first move compounds. Yes, shocking: causality has entered the chat.

What enterprises should copy is the instrumentation, not the benchmark

The tempting business conclusion is: use MatchTIR and your agents will improve. That may be true in some settings, but the more actionable conclusion is deeper.

To benefit from this style of training, an organization needs structured traces.

Without these assets, fine-grained credit assignment becomes guesswork. With them, the agent can be trained and audited at the level where business failures actually occur.

This is where the paper becomes relevant to workflow automation. Most enterprises do not need agents that philosophize about tools. They need agents that choose the correct system, pass the correct fields, recover from missing information, and avoid unnecessary operations. MatchTIR’s core idea aligns with that need: reward the steps that make the workflow succeed, not the general vibe of the transcript.

There is also a model-size implication. The paper’s 4B result suggests that better supervision can help smaller models compete with larger baselines. For businesses, that points toward a practical trade-off: before paying for bigger models everywhere, improve the quality of the training signal and trace evaluation. Bigger models may still win. But size is an expensive substitute for discipline.

The boundary is ground truth, and it is not a footnote

The paper’s main limitation is not cosmetic. MatchTIR relies on ground-truth tool trajectories for turn-level supervision. That assumption is reasonable in some environments and fragile in others.

It is reasonable when workflows are structured: invoice processing, report generation, internal data lookup, compliance checks, order status updates, CRM enrichment, or software tools with defined APIs. In these cases, there may be a known correct sequence or at least a verifiable reference sequence.

It is fragile in open-ended research. The authors explicitly note that tasks such as deep research are difficult because ground-truth parameters and reasoning steps are hard to determine. There may be many valid search paths, many useful intermediate sources, and no single correct trace. In that world, strict matching can punish legitimate alternatives.

There are also scale boundaries. The experiments use Qwen3-4B and Qwen3-8B. The paper does not show whether the same gains persist, shrink, or change shape for much larger models. It also evaluates through benchmarks, not production deployments with messy permissions, unreliable APIs, changing schemas, and humans who submit requests like “please do the usual thing from last month.”

Those boundaries do not weaken the paper. They prevent the wrong adoption story. MatchTIR is not a universal agent-training wand. It is a strong idea for environments where tool use is structured enough to verify.

The real lesson is that agents need accounting systems

The best way to read MatchTIR is not as a new reward function in isolation. It is an accounting system for agent behavior.

The old system says: the final answer passed; everyone gets paid.

MatchTIR says: show me which call matched the reference, which parameter was correct, which duplicate added no value, which turn affected later success, and how this rollout compares with other possible rollouts for the same task.

That is a much better accounting system. It also sounds suspiciously like what businesses should have wanted from agents all along.

For enterprise AI, the implication is straightforward. The next bottleneck is not only model intelligence. It is supervision granularity. If agents are going to operate across real systems, we need training pipelines that can distinguish the clean workflow from the lucky mess.

MatchTIR does not solve every version of that problem. It solves a specific, structured version well enough to make the broader point hard to ignore: when the work is multi-step, the salary should not be uniform.

Cognaptus: Automate the Present, Incubate the Future.