Retrieval

How to build a lightweight retrieval-augmented knowledge tool with grounded answers, source citations, narrow scope, and a realistic MVP.

TL;DR for operators A hybrid model can contain relatively little attention and still depend on that attention for its decisive retrieval step. Across 23 measured depth profiles from twelve checkpoints, the intermediate clue needed for a two-hop answer becomes most causally usable immediately before the layer that performs the largest conversion toward the final answer. That conversion occurs in an attention-containing layer in every measured profile. ...

TL;DR for operators A long-running assistant can store the right fact and still fail to retrieve it because that fact was represented alongside several unrelated meanings. AutoViewMem1 addresses this before search begins: it learns complementary ways to interpret a user’s history, writes structured memories under those views, removes redundancy, and then uses ordinary top-K vector retrieval. ...

TL;DR for operators A retrieval system can improve while its benchmark understates the improvement. A generated answer can remain consistent with its evidence while a lexical-overlap metric gives it a poor groundedness score. That is the central operational lesson from Morales, Hegde, and Syed’s urban-discourse system.1 Dense semantic retrieval beat TF-IDF across every reported held-out retrieval metric, including Precision@10 of 0.660 versus 0.545. But when the dense system’s actual retrieved chunks were rated directly, its Precision@10 was 0.94. The evaluation unit changed, and so did the apparent quality of the same retriever. ...

TL;DR for operators Running a sequence model in linear time does not settle the memory-design problem. A recurrent model still has to decide what shape its compressed history should take. RunningTensor1 replaces the matrix memory used by conventional linear attention with a higher-order tensor state. In the paper’s clearest matched ablation, an order-3 tensor state with the same ConcOnlyC projection averages 98.5 F1 on multi-query associative recall, versus 30.6 F1 for the matrix-state version. That gap makes memory geometry a serious architecture variable for workloads dominated by recovering many earlier associations. ...

TL;DR for operators When a system must map an ambiguous phrase in text to a canonical event in a knowledge base, enlarging the search pool is not necessarily the best use of compute. The more consequential failure may occur earlier: weak contextual evidence produces the wrong candidates, leaving the final linker nothing useful to rank. ...

TL;DR for operators A retrieved prompt can contain one useful sentence or evidence distributed across several passages. Giving both requests the same compressed-context budget assumes that the amount of information worth preserving is constant. The evidence in ATACompressor1 suggests that assumption is costly: on MSMARCO with LLaMA-2-7B, removing adaptive allocation reduced F1 from 50.06 to 47.52 and compression ratio from 27.36x to 19.06x; removing selective compression reduced F1 further to 40.83. ...

TL;DR for operators When a system receives hundreds of thousands of tokens, the expensive part is not only fitting the evidence into memory. It is deciding how much reasoning effort each part deserves. Processing every chunk with the same autoregressive budget preserves more opportunity to use evidence, but it also spends latency and GPU time on material that may never matter. ...

TL;DR for operators A RAG system can retrieve highly relevant passages and still train its generator on poor learning examples. Evidence that makes the answer nearly automatic may encourage shortcuts; evidence that leaves the answer effectively unreachable provides little usable training signal. BAR-RAG1 tests a different objective: keep evidence relevant, but prefer training sets on which the current generator succeeds roughly half the time. The selector is used to construct that training distribution, the generator is adapted to it, and the selector is then removed. Across three backbones, best-iteration BAR-RAG improves average exact match over standard RAG by 7.3, 12.2, and 11.6 points respectively. Against the stronger RAG-SFT baseline, the gains narrow to 3.9, 7.3, and 6.2 points. ...

TL;DR for operators A RAG system can fail even when retrieval looks superficially healthy: several top-ranked documents repeat the query vocabulary, while the document that actually contains the answer sits lower in the list. The paper studies a reranking rule designed specifically for that situation.1 Instead of rewarding more query-document overlap, it removes words that exactly or semantically echo the query and scores what remains. In an eight-theme controlled keyword-stuffing diagnostic, this semantic variant improves mean target rank from 2.88 to 1.25 and pushes stuffed distractors from mean rank 2.00 to 4.50. ...