Reranking

TL;DR for operators A RAG system can retrieve highly relevant passages and still train its generator on poor learning examples. Evidence that makes the answer nearly automatic may encourage shortcuts; evidence that leaves the answer effectively unreachable provides little usable training signal. BAR-RAG1 tests a different objective: keep evidence relevant, but prefer training sets on which the current generator succeeds roughly half the time. The selector is used to construct that training distribution, the generator is adapted to it, and the selector is then removed. Across three backbones, best-iteration BAR-RAG improves average exact match over standard RAG by 7.3, 12.2, and 11.6 points respectively. Against the stronger RAG-SFT baseline, the gains narrow to 3.9, 7.3, and 6.2 points. ...

TL;DR for operators A RAG system can fail even when retrieval looks superficially healthy: several top-ranked documents repeat the query vocabulary, while the document that actually contains the answer sits lower in the list. The paper studies a reranking rule designed specifically for that situation.1 Instead of rewarding more query-document overlap, it removes words that exactly or semantically echo the query and scores what remains. In an eight-theme controlled keyword-stuffing diagnostic, this semantic variant improves mean target rank from 2.88 to 1.25 and pushes stuffed distractors from mean rank 2.00 to 4.50. ...

Search is where nuance goes to die. A customer asks for a long evening dress, preferably not pink. A retrieval model sees “dress,” “evening,” perhaps “pink,” and returns something short, bright, and entirely wrong with the confidence of a clerk who has technically read the sentence but not understood the assignment. The business consequence is familiar: fewer conversions, more irrelevant recommendations, and yet another dashboard where “semantic relevance” looks respectable while customers quietly leave. ...