Search When It Hurts: How UR² Teaches Models to Retrieve Only When Needed

TL;DR for operators

UR² is a useful paper because it attacks the part of RAG that most demos politely ignore: search can make a model worse when it is used badly.¹ The framework trains smaller language models to coordinate retrieval and reasoning, rather than bolting a search box onto a chatbot and hoping the context window will behave itself. Hope, regrettably, is not a retrieval strategy.

The operational claim is simple. A model should not retrieve because retrieval exists. It should retrieve when the problem is hard enough, factual enough, or uncertain enough to justify the extra evidence. UR² tries to teach that behaviour through reinforcement learning, using a difficulty-aware curriculum, summarized evidence, and a two-stage reward design.

The paper’s most useful correction is aimed at a common business misconception: RAG is not improved by retrieving more documents. Uncontrolled retrieval can produce lazy or meaningless queries such as “which option is correct,” effectively outsourcing reasoning to a search system that cannot reason. At the other extreme, noisy raw documents can make the model stop retrieving at all, collapsing back into ordinary chain-of-thought.

The evidence is strongest for small-to-mid-sized models. UR² is tested on Qwen-2.5-3B, Qwen-2.5-7B, and LLaMA-3.1-8B across MMLU-Pro, medical QA, math, and open-domain QA. The headline Qwen-2.5-7B results include 53.3 average EM on MMLU-Pro, 69.6 on MedQA, 62.8 on out-of-domain MMLU-Pro Medical, 71.0 average math score, and 58.5 average F1 on open-domain QA. The exact Medicine aggregate is slightly inconsistent across the paper’s prose and table, so the safer interpretation is the component result: UR² materially improves both in-domain MedQA and out-of-domain medical evaluation.

For enterprise systems, the lesson is retrieval governance. Search should be treated as a learned operational decision: when to call it, what to ask, which evidence to pass forward, and when to stop. That matters for customer support, compliance review, research agents, legal triage, technical support, medical knowledge assistance, and any workflow where blindly stuffing documents into a prompt is expensive theatre with a progress bar.

The boundary is also clear. The work is benchmark-based. The main experiments are on models up to 8B parameters. Online retrieval is tested, but not fully used during training. The summarization layer helps, but also becomes a new component that must be monitored. UR² is not a universal recipe for “agentic search.” It is a disciplined answer to a narrower and more valuable question: how do we stop small models from searching like interns on their third coffee?

Search is useful only after the model learns shame

The standard enterprise RAG story is tidy. The model does not know something, so it retrieves documents. The documents ground the answer. Hallucination goes down. Everyone updates the roadmap deck.

UR² starts from a less flattering observation: retrieval does not automatically make reasoning better. Sometimes it gives the model a crutch. Sometimes it gives the model a pile of noisy text and calls it knowledge. Sometimes the model learns that search is safer than thinking, and sometimes it learns the opposite—that search is so messy it should be ignored.

That is the paper’s central mechanism-first contribution. UR² is not primarily a new retriever, a new vector database arrangement, or another “put relevant chunks in context” method. It is a training framework for balancing two behaviours that pull against each other:

That distinction matters. Most RAG implementations focus on the pipeline: embed, retrieve, rank, inject, generate. UR² focuses on behaviour: when should the model decide that search is warranted, and how should it behave after evidence arrives?

The paper’s answer is three-part. First, retrieval should be activated selectively based on difficulty. Second, retrieved evidence should be summarized into compact, reasoning-compatible form before the policy model uses it. Third, the model should be trained in stages: first to use the retrieval tool correctly, then to optimize answer quality.

The result is less glamorous than a giant “deep research” agent and more useful: a model that learns not to search unless the situation deserves it.

The real enemy is the retrieval–reasoning imbalance

UR² frames the problem as an imbalance between retrieval and reasoning. This is the right lens, because the failure is not merely about bad search results. It is about bad incentives.

If a model is allowed to retrieve freely during reinforcement learning, it can discover a shortcut. Instead of reasoning through a multiple-choice problem, it may issue a vague query that asks the search system to do the work. The paper gives the flavour of this behaviour with trivial queries such as “which option is correct.” That query is not retrieval. It is a confession.

But the opposite failure is just as important. If the model receives raw or noisy retrieved documents, the search process becomes cognitively expensive. The model has to sift irrelevant text, reconcile contradictions, and decide what to trust. During RL, that can push the model away from retrieval entirely. It collapses into pure chain-of-thought, even when the task actually needs external knowledge.

So UR² is designed around a behavioural constraint: search must be easy enough to use when useful, but costly enough not to become a reflex.

That gives the paper a more practical contribution than the usual “RAG improves accuracy” claim. The enterprise version is familiar. Teams add retrieval to a support bot, legal assistant, or internal knowledge agent. Then they discover three pathologies:

1. The system searches for everything, driving latency and cost up.
2. It retrieves noisy or irrelevant documents, making answers more verbose but not more correct.
3. It performs well in demos and poorly on edge cases, because the retrieval policy is just prompt etiquette wearing a lab coat.

UR² does not solve every one of those problems. It does, however, give a clearer design principle: retrieval should be trained as a controlled action, not treated as a default preprocessing step.

UR² teaches search as a staged skill, not a prompt habit

The first design move is a difficulty-aware curriculum.

UR² sorts training examples by difficulty using rollout performance from Qwen-2.5-7B-Instruct. Each question receives multiple rollouts, and the average performance score is used to categorize it as easy, medium, or hard. Extremely difficult samples are filtered out because they destabilize learning. The retained data are sampled with a hard-heavy ratio: 7:2:1 for hard, medium, and easy questions.

This is not just dataset housekeeping. It is behavioural shaping.

For math tasks, only hard problems receive retrieval-augmented prompting. Easy and medium problems use direct step-by-step reasoning. For open-domain QA, retrieval is used consistently because the task is inherently knowledge-dependent. For multiple-choice reasoning and medical tasks, the paper adapts retrieval activation to domain needs.

That design tries to prevent the model from learning the wrong lesson. If every problem includes search, the model may infer that search is part of the answer format. If no problem includes search, it never learns to use evidence when its internal knowledge is weak. UR² instead teaches a conditional policy: retrieve when the problem type and difficulty justify it.

The second design move is LLM-summarized retrieval.

Rather than passing raw retrieved documents directly to the policy model, UR² uses an LLM summarizer to convert retrieved material into shorter, relevant evidence. This does two things. It reduces noise, which helps prevent retrieval avoidance. It also reduces context length, which matters for latency and compute.

The compression numbers are not decorative. In offline retrieval, the paper reports average retrieved content falling from 1,291 tokens to 128 tokens, a compression ratio of 7.87. In online retrieval, where documents are longer and messier, the average falls from 11,346 tokens to 761 tokens, a compression ratio of 14.91. This is the kind of engineering detail that should make operators pay attention. Summarization is not merely a quality layer; it is a cost-control layer.

The third design move is two-stage optimization.

Stage 1 is retrieval activation. The goal is not primarily answer accuracy. The model is rewarded for using the retrieval tool correctly: obeying format constraints, issuing valid queries, and avoiding fallback failures. For some settings, this stage is brief—on the order of 10 steps for the Qwen-2.5-7B example described in the method.

Stage 2 is answer quality optimization. Once the model can invoke retrieval properly, correctness enters the reward. The model is then trained to answer well while preserving the retrieval behaviour learned in Stage 1.

This staging is important because tool use and answer correctness are entangled. If the model is punished only for wrong answers before it understands the tool, it may decide the tool is dangerous and stop using it. If it is rewarded only for tool use, it may learn to search theatrically. Stage 1 teaches the handle; Stage 2 teaches the judgment. Very basic pedagogy, somehow still underused.

The prompt rules are really behavioural guardrails

UR²’s retrieval prompting is strict. The model is told to issue concise queries under a word limit, target only one fact or statement per query, and use a maximum number of queries. The exact limits vary by task: multiple-choice and math prompts allow up to four queries, while open-domain QA allows up to five.

For math, the prompt explicitly forbids search for reasoning-related content such as whether a solution approach is valid or how to compute a specific value. It allows search for definitions, formulas, theorems, or constants. That distinction is small but important. Search can provide a formula; it should not be asked to solve the problem.

For enterprise deployment, this is a useful pattern. A retrieval policy should distinguish between:

The paper’s prompts are not a finished product interface. They are a training scaffold. But the scaffold exposes a business rule: retrieval calls should be auditable, narrow, and evidence-seeking. If an enterprise agent cannot explain why it searched, what fact it sought, and how the evidence changed the answer, the retrieval system is not governed. It is merely busy.

The main results show breadth, not just another QA bump

UR²’s experimental setup is broader than many retrieval-RL papers. It evaluates across four task families: mathematical reasoning, medical QA, MMLU-Pro-style reasoning, and open-domain QA. The models are Qwen-2.5-3B, Qwen-2.5-7B, and LLaMA-3.1-8B. The baselines include plain chain-of-thought, standard RAG, advanced prompt-based RAG methods, CoT-RL methods, and RAG-RL methods such as Search-R1 and R1-Searcher.

The strongest practical evidence is not that UR² wins every row by a spectacular margin. It does not. The evidence is that the same framework improves across different task types where retrieval has different roles.

The math result deserves restraint. UR² improves the Qwen-2.5-7B math average from 68.8 under vanilla RL to 71.0, but this is not the main miracle. Math problems usually need computation more than retrieval. The more relevant point is that UR² does not destroy math reasoning while adding retrieval. That is a quieter, better result.

Open-domain QA is different. Retrieval is central there, and UR² is operating in the natural habitat of RAG-RL methods. The gains over strong RAG-RL baselines are modest but meaningful: 58.5 average F1 for Qwen-2.5-7B versus 56.1 for Search-R1. On Qwen-2.5-3B, UR² reaches 55.3 average F1, ahead of Search-R1 at 47.1. The small model result is more interesting because it suggests retrieval coordination can compensate for weaker parametric knowledge.

The cross-model evidence is mixed in exactly the way one would expect from real RL training. Qwen models show stronger and more stable gains. LLaMA-3.1-8B improves, but the paper notes less stable training dynamics and early saturation. That is not a fatal flaw. It is a useful warning: retrieval policy learning is model-family dependent. No one gets a free lunch; some just invoice it as “foundation model variance.”

The ablations are where the paper earns its mechanism

The main tables show that UR² works. The ablations explain why it works. That distinction matters because without ablations, this paper would risk becoming another benchmark trophy shelf.

The summarization ablation is the sharpest. In the main reasoning ablation, removing LLM summarization leads to training failure: the model abandons retrieval and collapses into pure reasoning. That supports the paper’s diagnosis that retrieval noise is not a minor inconvenience. It changes the learned policy.

The nuance is that open-domain QA behaves differently. In the appendix, UR² without summaries still performs competitively in some open-domain settings, though summaries improve average F1 materially across corpus configurations. This makes sense. Open-domain QA is already retrieval-native. The model expects evidence. In reasoning-heavy tasks, retrieval is optional and therefore easier to abandon when evidence becomes unpleasant.

Stage 1 also matters. Removing it drops Qwen-2.5-7B MMLU-Pro performance from 53.3 to 49.5 in the main ablation table. That is not a tiny formatting issue. It means the model needs a cold-start phase for retrieval behaviour before answer accuracy becomes the dominant reward.

Task mixing is subtler. The “without task mixing” variant retrieves for all samples and shows only minor accuracy changes. The practical interpretation is not “task mixing is useless.” It is that task mixing is partly an efficiency and behaviour-control mechanism. If similar accuracy can be achieved with fewer unnecessary retrieval calls, the business value shows up in latency, token cost, and operational predictability rather than a dramatic benchmark delta.

Retrieval frequency proves the model learned a policy, not a ritual

One of the more useful analysis results is the retrieval frequency table. UR² averages 0.94 retrieval calls per query on Math500, 1.34 on MMLU-Pro, and 2.46 on open-domain QA.

That pattern is exactly what one would want. Math needs mostly internal reasoning, with occasional factual lookup. MMLU-Pro sits in the middle: some questions require knowledge verification, some require reasoning. Open-domain QA needs frequent external evidence.

This result is not just a cute behavioural metric. It is evidence that UR² is not following a fixed retrieval script. A fixed script would retrieve at roughly similar rates across tasks. UR² instead adjusts retrieval intensity to task structure.

For enterprise teams, retrieval frequency should be a first-class observability metric. A support agent that retrieves ten documents for every password reset question is broken. A compliance agent that never retrieves policy text is also broken. The target is not “more retrieval.” The target is calibrated retrieval.

A useful production dashboard would track:

UR² does not provide a production monitoring system. But it gives the behavioural variables such a system should watch.

Online search is promising, but it is not the same as training in the wild

The paper includes an online retrieval evaluation. This is important because local Wikipedia-style corpora are cleaner than the web, and the web is where retrieval systems go to become confused in public.

UR² is trained with offline corpora but evaluated with an online search pipeline using web search, crawling, HTML-to-Markdown conversion, and summarization. The online results are encouraging. For Qwen-2.5-7B, online search improves MMLU-Pro average from 53.3 to 55.4 and open-domain QA average F1 from 58.5 to 61.6. It improves 2WikiMultiHopQA from 62.6 to 75.8 F1 and Bamboogle from 64.5 to 73.7 F1.

There are exceptions. Math falls slightly from 71.0 to 70.0, which is unsurprising because search is not the main bottleneck. HotpotQA drops from 71.2 to 62.0 F1, which the paper attributes partly to rate limits blocking access to many gold Wikipedia pages.

This should be read as a robustness test, not a final deployment proof. The online setting shows that learned retrieval behaviour transfers beyond the local corpus. It does not show that UR² has mastered the full instability of live enterprise knowledge environments: access permissions, stale documents, conflicting internal policies, duplicated pages, low-quality web content, and all the other delightful ways information systems express themselves.

Still, the result matters. Training a model to search selectively on a controlled corpus can produce useful behaviour under noisier online retrieval. That is operationally attractive because fully training inside live retrieval environments is expensive, slow, and hard to reproduce.

The business value is retrieval governance, not chatbot cleverness

The immediate business temptation is to read UR² as “small models plus RAG can compete with bigger models.” There is some truth there. The paper reports that Qwen-2.5-7B with UR² reaches competitive results against GPT-4o-mini and GPT-4.1-mini on several benchmarks. That is relevant for cost-sensitive deployments.

But the deeper business value is governance.

UR² suggests a practical architecture for enterprise AI systems:

User task
  ↓
Difficulty / knowledge-dependence assessment
  ↓
Direct reasoning if sufficient
  ↓
Targeted retrieval if needed
  ↓
Evidence summarization and compression
  ↓
Reasoning with evidence
  ↓
Answer plus retrieval trace

The most important part is not the retriever. It is the decision boundary before retrieval.

In many enterprise workflows, the expensive mistake is not that the model lacks access to information. It is that the model uses access poorly. It retrieves irrelevant material, buries the answer in context, or treats retrieved text as truth without enough scrutiny. UR² pushes toward a cleaner separation:

This is especially relevant for smaller models. Large proprietary models may handle simple retrieval prompts reasonably well because their internal reasoning and instruction-following are strong. Smaller models need more structure. UR²’s results indicate that reinforcement learning can teach some of that structure rather than relying on prompt engineering alone.

The ROI case is therefore not just lower model cost. It is lower retrieval waste, lower latency, better evidence discipline, and more predictable behaviour under task variation. That is less exciting than “autonomous research agent,” but it is far closer to something procurement can understand without requiring mood lighting.

What Cognaptus would infer—and what the paper does not directly show

The paper directly shows that UR² improves benchmark performance across several task families for 3B–8B models, using a combination of difficulty-aware training, summarized retrieval, and staged RL optimization. It also shows that retrieval frequency adapts by task type, that summarization stabilizes retrieval-heavy reasoning, and that learned retrieval behaviour can transfer to online search evaluation.

Cognaptus would infer three business lessons.

First, retrieval should be budgeted dynamically. Not every user query deserves search. A model that retrieves selectively can reduce context cost and latency without necessarily sacrificing quality.

Second, summarization is not optional plumbing in noisy retrieval environments. It is a control layer. Its job is not merely to shorten documents, but to make evidence usable by the reasoning model. That means summarizers need their own evaluation: factual preservation, omission risk, compression ratio, and domain robustness.

Third, tool-use training matters more for small models than for large ones. A 7B or 3B model may not reliably infer how to use retrieval from prompts alone. UR²’s staged training gives these models procedural competence: invoke the tool correctly first, optimize answer quality later.

Now the boundaries.

The paper’s main experiments stop at 8B models, with only a preliminary larger-model note. That is enough to support the small-model thesis, but not enough to generalize confidently to 14B, 32B, or frontier-scale systems.

The evaluation is benchmark-heavy. MMLU-Pro, MedQA, MATH500, Minerva, HotpotQA, 2WikiMultiHopQA, Bamboogle, and MusiQue are useful, but enterprise environments add access control, versioned documents, confidential corpora, compliance requirements, and user-specific context. Those are not minor implementation details; they are where many RAG systems go to discover character.

The summarizer is a dependency. The paper includes encouraging ablations with weaker and distilled summarizers, but main configurations still rely on strong summarization in several places. A poor summarizer can delete crucial evidence, over-compress nuance, or turn ambiguity into false clarity. Summarization is a seatbelt, not a force field.

The online search results are an extension, not a complete training regime. UR²’s online evaluation is promising, but future work is still needed on training with online corpora and raw web complexity.

Finally, RL reproducibility remains a constraint. The paper reports stable evaluation on a targeted repeated HotpotQA run, but also notes that multiple full evaluation runs were constrained by API cost. That is understandable. It is also a reminder that production adoption should replicate results on the organization’s own tasks before celebrating.

Search should be a decision, not a reflex

UR² is valuable because it makes retrieval less magical and more managerial.

It does not say: give every model a search tool and watch intelligence emerge. It says: teach the model when search is worth using, make the evidence digestible, reward tool use separately from final correctness, and keep direct reasoning alive.

That is a better foundation for enterprise AI than the usual RAG optimism. Retrieval is not knowledge. It is access to possible evidence. The model still has to decide what to ask, what to trust, and when to stop asking. UR² shows that those behaviours can be trained, at least for smaller models and controlled benchmark settings.

The lesson for operators is blunt. If your AI system searches constantly, it is not diligent. It may be confused. If it never searches, it is not efficient. It may be overconfident. The useful system is the one that searches when search hurts less than ignorance.

That, finally, is the real promise of UR²: not more retrieval, but better hesitation.

Cognaptus: Automate the Present, Incubate the Future.