Predicting More Can Teach an Agent Less

TL;DR for operators

An assistant searching documents or navigating a shopping website encounters far more information than it needs for its next action. Training it to predict everything it observes may therefore impose unnecessary learning demands.

In one reported comparison, adding full-observation prediction to reinforcement learning reduced average multi-hop question-answering accuracy from 18.88% to 17.71%. A method that selected observation content instead raised accuracy to 22.06%.

That method, Decision-Guided Selective World Modeling (DGS-WM), comes from Model What Matters: Decision-Guided Selective World Modeling for Language Agents.¹ It selects prediction targets according to their support for the agent’s subsequent response and assigns less training weight to targets the model already predicts confidently.

For teams training retrieval assistants or web-operation agents, this suggests testing selective prediction against complete tool-response prediction. The agent still receives full observations when operating; selection changes its auxiliary training objective. The reported gains are promising, but training costs, relevance reliability, and performance in live enterprise workflows remain unestablished.

Why predicting the complete observation can underperform

Reinforcement learning commonly trains agents using rewards associated with task outcomes. An agent may receive a reward for answering correctly or completing a purchase, but relatively little direct supervision about the environmental information encountered between actions.

Policy-world modeling co-training adds another objective: predict the observation following an action. Both the task-reward objective and observation-prediction objective update the same language model.

The assumption is reasonable. If an agent learns what its actions produce, it may become better at deciding what to do next. Yet a retrieved document can contain relevant evidence alongside unrelated paragraphs, while a product page can contain useful specifications alongside navigation elements and other content.

Predicting every part of these observations gives auxiliary learning weight to information that may contribute little to task performance.

The paper tests this distinction against GRPO-only reinforcement learning and ECHO, a method that adds full-observation world-modeling supervision.

Source: Section 4.2, Table 1. QA figures are Exact Match percentages, macro-averaged across three single-hop and four multi-hop benchmarks. WebShop reports task success.

The smaller model illustrates the problem particularly clearly. ECHO improves WebShop performance but slightly reduces both QA averages relative to GRPO alone. DGS-WM improves all three outcomes.

For Qwen3-8B, full-observation modeling already improves QA accuracy, but DGS-WM adds another 1.39 percentage points in single-hop and 2.33 points in multi-hop accuracy over ECHO.

Thus, the finding is not that predicting more information always harms learning. Rather, the choice of what to predict affects whether auxiliary supervision improves the task policy.

How the next response determines what gets predicted

DGS-WM changes training at two levels: which environmental content becomes a prediction target, and how strongly that target contributes to the loss.

First, it divides each observation into semantically meaningful units. Retrieved QA documents are partitioned into sentences. WebShop pages are partitioned using task-aware interface structures, such as semantic webpage blocks.

Each candidate unit is then evaluated against the agent’s actual next response in its recorded training trajectory.

The method asks: given the preceding interaction and action, how likely would the sampling policy have been to produce that response if it received this particular observation unit?

A higher average token log-likelihood indicates stronger support for the recorded response.

This is Decision-Guided Observation Selection (DGOS). Candidates are retained only when their support scores exceed the observation’s mean score plus a threshold determined by its score standard deviation. The main experiments use a threshold coefficient of 0.5.

Selected units are concatenated in their original order. The model then learns to predict this selected content from the preceding history and executed action. If nothing qualifies, auxiliary observation prediction is skipped for that interaction.

Importantly, no manually labeled relevance judgments or separate teacher model are required for this selection process.

Nor does the system remove unselected information from the agent’s ordinary interaction context. The full observation remains available to the policy; only the additional prediction objective is selective.

Confidence weighting changes how much each target matters

Selection alone does not resolve another allocation problem: some selected targets are already easy for the model to predict.

DGS-WM therefore introduces Confidence-Adaptive Weighting (CAW). It estimates confidence from the selected target’s average prediction loss and reduces the auxiliary weight assigned to confidently predicted targets.

In the main methodology, the target weight is:

Here, $q_t$ represents prediction confidence, and $\gamma$ determines how strongly the weight decreases with confidence. The main experiments use $\gamma=1$.

The detached weight prevents gradients from propagating through the weighting calculation itself. The weighted prediction loss is then combined with the reinforcement-learning objective, using an auxiliary-loss coefficient of 0.02 in the main experiments.

The two components address different decisions: observation selection determines the material to learn, while confidence weighting determines how much additional supervision that material receives.

The distinction is supported by component ablations.

On Qwen3-1.7B, removing observation selection reduces multi-hop average Exact Match from 22.06% to 20.31%. Removing confidence weighting, while retaining selection, reduces it to 21.66%.

Single-hop performance falls from 45.77% to 44.45% without selection and to 44.82% without confidence weighting.

These ablations indicate that both components contribute, with the larger measured loss arising from removing selection in this configuration.

The gains extend beyond one model and optimizer

The primary comparisons use two Qwen3 model sizes, but the authors also evaluate Qwen2.5-1.5B, Llama-3.2-3B, and Qwen2.5-7B under GRPO. Further experiments test PPO, RLOO, and GMPO with Qwen2.5-1.5B.

Across all twelve reported task-category-by-configuration comparisons, DGS-WM exceeds the corresponding RL-only and ECHO results.

These are robustness comparisons, not separate evidence that the selection mechanism identifies causally necessary information. The ablations more directly test the contribution of each training component, while sensitivity experiments identify the best reported settings among the examined values of the selection threshold and confidence exponent.

Four illustrative training-log examples also show selected content occupying only 7.4–12.5% of the original observations. Those examples help explain the filtering mechanism but do not establish a benchmark-wide compression rate.

What enterprise agent teams can test

For a team training a retrieval-based assistant, the immediate decision concerns auxiliary supervision design.

If search results frequently contain long documents with limited answer-bearing content, selective sentence prediction could direct more learning toward information associated with subsequent answers. A relevant evaluation would compare RL-only training, complete-document prediction, and DGS-WM using the same base model, task distribution, and evaluation protocol.

For developers of transactional web agents, semantic interface components provide a possible prediction unit. Product specifications, transaction details, and confirmation messages might be preferable targets to complete webpages, provided the partitioning preserves information required for safe actions.

These are Cognaptus inferences from the benchmark findings, not demonstrated enterprise outcomes.

One deployment-relevant advantage is architectural: selection and confidence weighting occur during training. The resulting policy does not require an additional world-modeling stage at inference.

However, training-time scoring requires evaluating candidate observation units against subsequent responses. The paper does not quantify the resulting computational overhead. Any implementation decision should therefore measure end-task accuracy, training cost, selection stability, and failure rates rather than assuming reduced prediction volume implies lower total training cost.

What the selected information does not prove

The most consequential limitation concerns the definition of relevance.

A candidate receives a high score when it supports the response the policy actually produced. That response may be wrong, and strong predictive association does not establish that the candidate caused the decision.

Consequently, the method could prioritize information consistent with an imperfect existing policy. Treating its scores as causal explanations of agent behavior would require separate counterfactual or intervention-based evidence.

Observation segmentation presents another boundary. The experiments use sentence splitting for QA and handcrafted structural rules for WebShop. Dynamically changing enterprise interfaces, complex documents, and multimodal observations may require different segmentation strategies.

The paper also reports benchmark point estimates without confidence intervals or statistical significance tests. Production generalization cannot be inferred from consistent reported gains alone.

Finally, a reproduction detail deserves attention: the main equations and algorithm agree on the confidence-weight expression, but Appendix A.2.3 describes a different expression. Implementations should explicitly resolve that discrepancy rather than assuming the descriptions are interchangeable.

The training objective should reflect the decision

DGS-WM demonstrates that stronger language-agent training need not require predicting every available environmental detail. Across the evaluated settings, selecting content associated with subsequent responses and weighting it by prediction confidence produces better task performance than the tested alternatives.

For operational teams, the next step is a controlled training comparison: determine whether selective auxiliary prediction improves the intended agent’s decisions enough to justify its segmentation and scoring costs.

The relevant objective is better task performance, not more comprehensive reproduction of everything the agent has observed.

Cognaptus: Automate the Present, Incubate the Future.