More Retrieval Is Not Free: Price Every RAG Component Before You Ship It
TL;DR for operators When a knowledge-grounded assistant needs improvement, adding another retrieval stage is not automatically the safest use of inference budget. In this study, the most expensive retrieval option in the matched comparison—combining semantic and keyword-based search—reduced accuracy by 1.85 percentage points relative to dense retrieval while adding 3,079.69 seconds of runtime across the evaluation run. More machinery produced a worse benchmark result. ...