Recall Is Not a Retrieval Strategy: Choosing GraphRAG or VectorRAG by Workload

TL;DR for operators

A technical assistant searching thousands of papers has two competing needs: preserve enough surrounding text to understand experimental context, and connect facts that are distributed across papers. A single retrieval score can make that choice look simpler than it is.

In Gupta et al.’s Polymer Literature Scholar study,¹ graph-based retrieval achieved full-corpus recall of 0.903–0.938, versus 0.717 for dense text retrieval. Yet answer accuracy remained close: 0.964–0.973 for GraphRAG and 0.960 for VectorRAG. The retriever that was substantially better at recovering designated evidence was only modestly better at producing correct answers.

For technical teams, that changes the architecture decision. GraphRAG is the stronger candidate when the workload depends on relationships across studies, terminology normalization, provenance, or multi-hop synthesis. VectorRAG remains attractive when the answer depends on detailed passages containing mechanism, experimental conditions, or quantitative context. Neither choice should be standardized from recall alone.

The recall gap is large; the answer-quality gap is not

The paper builds two retrieval pipelines over a manually verified corpus of 1,028 full-text papers on polyhydroxyalkanoates and related biodegradable polymers. The corpus contains 44,609 parsed paragraphs, consolidated into 10,665 context-preserving chunks for dense retrieval.

VectorRAG searches semantically similar text and returns surrounding narrative context. GraphRAG instead extracts scientific entities and relationships, canonicalizes terminology, and retrieves relational paths using a mixture of string matching, semantic entity matching, and cross-encoder reranking.

The most consequential result appears only after the system is tested at full-corpus scale.

The difference matters because the paper’s Recall@K metric asks whether the designated ground-truth paragraph appears among the retrieved results. It does not ask whether another relevant passage could support the same answer.

That distinction helps explain why VectorRAG can miss the benchmark’s expected paragraph and still generate a grounded answer. The authors explicitly caution that exact retrieval recall is therefore an incomplete measure of scientific usefulness. Even their more permissive paper-level retrieval metric cannot determine whether the retrieved evidence is scientifically sufficient.

For an operator evaluating RAG, a retrieval miss and an answer failure are not interchangeable events.

Small benchmarks can hide the architecture decision

On the controlled 21-paper subset, the choice looks much less consequential. The benchmark contains 113 manually written questions and 943 paragraphs. GraphRAG recall ranges from 0.982 to 0.991, while VectorRAG reaches 0.973. Answer accuracy is 0.991 for GraphRAG and 1.000 for VectorRAG.

This controlled evaluation is best read as evidence that both implementations can work when the search space is constrained. It is not an ablation showing that architecture stops mattering.

When the corpus expands to 1,028 papers, GraphRAG recall remains above 0.90 while VectorRAG falls to 0.717. The likely mechanism is representational. Dense retrieval must distinguish semantically similar passages across a much larger textual space. GraphRAG narrows retrieval through canonical entities and explicit relations, allowing differently worded references to the same scientific concept to converge on a shared representation.

That is a scaling result, not proof that graphs produce better answers for every task.

GraphRAG and VectorRAG preserve different evidence

The paper’s representative-query analysis explains why a universal winner would be the wrong conclusion.

GraphRAG is strongest when the information needed to answer a question is relational: entities have to be connected, evidence aggregated across papers, or a multi-step scientific relationship reconstructed. Its retrieved tuples also expose an explicit evidence trail, making it easier to inspect which relationships supported an answer.

VectorRAG preserves more of the original passage. For questions where the relevant information is concentrated in a few paragraphs, that can produce richer mechanistic explanations and retain quantitative or experimental detail that a graph representation may compress away.

The failure modes differ accordingly. Dense retrieval carries redundant text and higher token consumption and loses exact retrieval recall as the corpus grows. Graph retrieval can instead become incomplete when the extracted graph or retrieved subgraph is sparse. In those cases, the paper reports that the system tended to stay within retrieved evidence or decline to answer rather than fill missing links speculatively.

The latter is operationally relevant, but its evidentiary boundary matters: the abstention finding comes from benchmark behavior, representative cases, and expert assessment, not from a separate large-scale hallucination-rate experiment.

Deployment should optimize a bundle of outcomes

Cognaptus’ inference from the benchmark is that retriever selection should begin with the information shape of the workload, then be tested against operational constraints.

Architecture choice alone is still insufficient. Latency in the reported results is model-dependent: with GPT-4o-mini, VectorRAG is faster at full scale, at 17.69 seconds versus 34.14 seconds for GraphRAG; with Llama-3.1-70B, GraphRAG is faster, at 24.18 versus 40.35 seconds. The paper’s broader narrative about GraphRAG latency should therefore not be generalized beyond the model-specific table results.

The GPT-4o-mini configurations also show lower reported average inference cost for GraphRAG, $0.0007 versus $0.0018 per query, but those numbers belong to this implementation and workload. They are not a general cost model for graph retrieval.

The deployment scorecard should consequently include answer accuracy, retrieval behavior, latency, inference cost, evidence traceability, contextual coverage, and the system’s behavior when evidence is insufficient.

Competitive expert scores do not make this a universal benchmark

A domain chemist compared five retrieval-generation systems across general, paper-specific, and multi-paper questions. GraphRAG with GPT-4o-mini and ChatGPT-5 with Web Search received the highest overall assessments, averaging between nine and ten across the evaluated categories.

That result supports the feasibility of a carefully curated domain RAG system competing with strong web-connected systems on expert-assessed scientific questions. It does not establish controlled equivalence between them. The commercial systems did not operate over exactly the same retrieval corpus, and the qualitative comparison used one domain chemist.

Generalization is the larger unresolved issue. The experiments are confined to PHA literature. The proposed mechanism—canonicalization helping fragmented terminology and graph structure helping relational retrieval—is plausible for other technical fields, but the paper does not benchmark those fields.

There is also a reproducibility detail worth retaining: the paper reports Qwen3-Embedding-4B dimensionality inconsistently, as 3,584 dimensions in Results and 2,560 in Methods. That discrepancy does not overturn the comparative benchmark, but it should be resolved by anyone reproducing the implementation.

Choose the evidence shape before the stack

The main contribution is not that GraphRAG scores higher on one retrieval metric. It is that retrieval architecture changes what evidence survives the search process.

At scale, GraphRAG preserves retrieval fidelity and makes relationships easier to inspect. VectorRAG preserves more textual context and can produce stronger answers when the relevant evidence lives inside detailed passages. Their answer accuracies remain close enough that recall cannot serve as the deployment decision by itself.

For R&D teams building technical assistants, the resulting decision is narrower and more testable: identify whether the workload primarily requires relational synthesis or contextual reading, then validate the candidate retriever on local questions using both retrieval and downstream answer measures. Add latency, cost, provenance, and abstention before deciding whether the additional retrieval machinery earns its place.

The PHA benchmark supplies a strong deployment hypothesis. Other technical domains still have to test it.

Cognaptus: Automate the Present, Incubate the Future.