Cite Before You Write: Agentic RAG That Picks Graph vs. Vector on the Fly

TL;DR for operators

Most enterprise RAG failures are not generation failures. They are retrieval-routing failures wearing a very convincing blazer.

The paper behind this article proposes an open-source agentic hybrid RAG framework for scientific literature review: bibliographic metadata and citation relationships go into a Neo4j knowledge graph; full-text PDF chunks go into a FAISS vector store; an LLM-based agent decides whether a user’s question should be answered through GraphRAG or VectorRAG; a Mistral-based generator produces the final answer; DPO is used to improve grounding; and bootstrap resampling is used to report evaluation uncertainty.¹

The operational lesson is simple but easy to miss: a literature-review assistant should not treat every question as “find semantically similar chunks and summarise them.” Some questions are relationship questions: who authored what, which papers cite which sources, which keywords connect to which publication years. Other questions are evidence questions: what does the full text say, how does a method work, what claim is supported by a passage. Mixing those up is how a system becomes fluent, expensive, and occasionally useless.

The reported results support that mechanism. Against a non-agentic baseline, the fine-tuned agentic model improves VectorRAG Context Recall by 0.63 and overall Context Precision by 0.56. It also improves VS Faithfulness by 0.24, VS Precision and KG Answer Relevance by 0.12 each, overall Faithfulness by 0.11, KG Context Recall by 0.05, and VS Answer Relevance plus overall Precision by 0.04 each. Those are not cosmetic deltas. They suggest that the system is retrieving more of the necessary evidence and wasting less context budget on irrelevant material.

But this is not a fully autonomous systematic-review machine, however much the phrase “agentic” would like to dress for the occasion. The benchmark contains 40 synthetic question-answer pairs, split evenly between vector and graph tasks, generated from a healthcare-focused multimodal LLM literature query covering 2023-2025. The paper itself notes several practical gaps: Cypher translation errors, scanned-document coverage, figures and tables, API variability, latency, and the need for human oversight. The right business takeaway is not “replace research teams.” It is “design research systems where retrieval choice is explicit before synthesis begins.”

The real move is not RAG. It is routing.

A familiar corporate research workflow goes like this. Someone asks for a “quick review” of a technical field. A knowledge team exports papers, dumps PDFs into a vector database, connects a chatbot, and hopes the answers look less suspicious than a generic LLM response. For a while, they do. Then the system is asked a question like: “Which papers published in 2024 cite the same foundational source as this one?” Or: “Which authors repeatedly appear across these subtopics?” Or: “What does the paper actually say about clinical deployment barriers?”

Those are not the same question.

The first two are graph questions. They require traversing structured relationships: publication year, author, citation, keyword, database source. The last one is a semantic evidence question. It requires reading full text, retrieving relevant passages, and grounding an answer in what those passages contain. A single vector search can sometimes bluff its way through both. Naturally, this is called flexibility. In production, it is often just confusion with embeddings.

The paper’s contribution is to make this choice an architectural decision. The pipeline creates two persistent memory systems from the same literature corpus. One is a knowledge graph, where papers connect to authors, publication years, citation DOIs, keywords, and source databases. The other is a vector store, where chunks from available full-text PDFs are embedded for semantic retrieval. A tool-using agent then chooses the retrieval function: Cypher-based GraphRAG for structured queries, or sparse-dense VectorRAG for content-heavy questions.

That choice is the heart of the paper. The generator matters. The DPO step matters. The bootstrap evaluation matters. But the strategic idea is that “retrieval” is not one thing. It is a portfolio of mechanisms, and the system should decide which mechanism matches the question before it starts composing an answer.

For enterprise research systems, that matters more than another polished summary interface. A system that can answer “what did the text say?” and “how are these records related?” through different retrieval paths is easier to audit, easier to debug, and easier to govern. It also has a smaller blast radius when something goes wrong. If the graph answer is wrong, inspect the graph schema or the Cypher translation. If the full-text answer is weak, inspect chunking, embeddings, reranking, or source coverage. That is boring engineering. Boring engineering is underrated, mostly by people who have not operated brittle AI systems.

The pipeline separates bibliographic structure from textual meaning

The framework begins with data collection from PubMed, arXiv, and Google Scholar APIs. The extracted metadata includes DOI, title, abstract, publication year, authors, PDF URL, and source database. Records with missing values are removed, and duplicates are filtered using DOI and title. The paper then extracts five keywords from each article using TF-IDF over the title and abstract, processes them through lemmatisation and lowercasing, and compares them with the initial search-query keywords using cosine similarity.

Only papers above the third quartile of cosine-similarity scores are retained. In plainer language: the pipeline keeps the top 25% of the retrieved literature by this relevance filter. This is not a universal best-practice law handed down from the heavens. It is a practical filtering heuristic. It narrows the corpus before storage and retrieval, which improves focus but can also exclude peripheral papers that later turn out to matter. Useful, yes. Omniscient, no.

After filtering, the system sends different parts of the literature into different stores:

The split is important because it prevents a common category error. Metadata should not be treated as merely another blob of text. It has structure. Structure deserves structure-aware retrieval. Similarly, full-text reasoning should not be squeezed into graph traversal unless the information has actually been extracted into graph form. Vector search exists for a reason; it is just not the only reason.

The implementation uses 2,024-character text chunks with a 50-character overlap, embedded with all-MiniLM-L6-v2 and indexed in FAISS. Vector retrieval combines BM25 keyword search with dense semantic search, then reranks candidate chunks with Cohere’s rerank-english-v3.0 model. Each method independently returns five candidate chunks before reranking compresses the final context.

This design is not exotic. That is part of its appeal. The paper is assembling known components into a routing architecture: Neo4j for relationships, FAISS for embeddings, BM25 for lexical matching, dense retrieval for semantic matching, reranking for context quality, LLaMA-3.3-70B-versatile for agentic tool choice, and Mistral-7B-Instruct-v0.3 for response generation. The novelty is less “we invented retrieval” and more “we stopped pretending one retrieval mode should answer everything.”

GraphRAG answers relationship questions; VectorRAG answers evidence questions

The paper’s GraphRAG path translates a natural-language query into Cypher, executes it against the Neo4j graph, and passes the retrieved results back to an LLM for formatting. The Cypher generator receives the graph schema and thirty input-output examples. This is classic few-shot prompting: show the model enough examples of natural language turning into valid graph queries, then hope it generalises without turning the database into interpretive dance.

GraphRAG is strongest when the answer lives in explicit relationships. For example: publication years, source databases, keyword associations, author links, citation edges, or indirect links through an intermediate node. These are questions where vector similarity can retrieve nearby prose but may not reliably perform exact relational traversal.

VectorRAG takes the other side of the work. It handles questions that depend on the content of papers rather than their graph position. It uses both sparse and dense retrieval, then reranks candidate chunks before passing them to the generator. This is more appropriate for questions like: “What problem does this method address?” or “What limitation does the paper identify?” The answer is not a node-edge pattern. It is in the text.

The agent sits between the user and these two retrieval systems. The authors use ten few-shot examples, five for each method, to guide tool selection. Each example includes the original question, selected retrieval tool, retrieved context, and final answer. That gives the router a small decision curriculum: when the question looks relational, call the graph; when it looks semantic, call the vector store.

Here is the mechanism in operational form:

User question
   |
   v
Agent decides retrieval mode
   |
   +--> GraphRAG: natural language -> Cypher -> Neo4j -> structured answer
   |
   +--> VectorRAG: BM25 + dense retrieval -> reranker -> full-text answer
   |
   v
Mistral generator formats grounded response

That diagram is the paper’s business relevance in miniature. The value does not come from letting an LLM “think harder.” It comes from reducing the number of decisions hidden inside a single black-box answer. Tool choice is visible. Storage mode is visible. Retrieval context is visible. Failure modes are at least theoretically inspectable. That is what separates a research assistant from a summarisation slot machine.

DPO tries to make the generator obey the retrieved evidence

The authors apply Direct Preference Optimization directly to the response generator using 15 human-annotated preference pairs. That is a small preference dataset, so it should not be read as a sweeping alignment result. The purpose is narrower: teach the generator to prefer answers that are grounded in retrieved context rather than answers that lean on the model’s internal memory.

This distinction matters. In RAG systems, the generator is often the final point of failure. Retrieval can fetch the right passage, but the model may still answer from prior knowledge, blend incompatible snippets, or produce a plausible synthesis that the retrieved context does not support. DPO is used here as a behavioural nudge: when context is available, use it.

The reported results suggest that this nudge helps, but not uniformly. Compared with the original agentic approach, the DPO-enhanced model improves KG Faithfulness by 0.19 and overall Faithfulness by 0.09. It also improves KG Context Recall by 0.08, KG Precision by 0.07, and KG Answer Relevance by 0.05. On the vector side, however, Context Recall remains unchanged, while Precision drops by 0.10, Faithfulness by 0.02, and Answer Relevance by 0.01.

That pattern is more interesting than a simple victory lap. DPO appears especially useful for structured-query tasks in this setup, but it does not magically improve every retrieval mode. On vector questions, the agentic system already retrieves relevant text; preference tuning may trade off some precision or alter answer behaviour in ways that do not improve every metric. In other words, DPO is not seasoning. You do not sprinkle it on a pipeline and declare the dish aligned.

For business deployment, the lesson is practical: preference tuning should be evaluated by task class. A compliance research assistant, patent-discovery tool, market-intelligence system, and clinical-evidence monitor may need different preference data. “Be grounded” is not specific enough. Grounded in what? A graph edge? A paragraph? A table? A regulator’s definition? A source hierarchy? The answer changes the training data.

The benchmark is designed to test routing, not full systematic review

The evaluation uses a synthetic benchmark because the authors state that no public benchmark exists for evaluating both VectorRAG and GraphRAG approaches in general domains. The test corpus is built from a search query around multimodal large language models and healthcare or medicine, limited to 2023-2025. From this corpus, the authors generate 40 question-answer pairs: 20 for VectorRAG and 20 for GraphRAG.

The VectorRAG questions are generated from randomly selected full-text chunks. The GraphRAG questions are built around five graph-question types: subject-centred questions, object discovery, predicate discovery, fact checking, and indirect relationship discovery. Each graph type receives four samples, giving 20 graph questions. The benchmark is then evaluated across three systems:

This is main evidence, not a full production simulation. The benchmark is balanced by design: half graph-specific, half vector-specific. That makes it a useful test of whether the system can exploit retrieval specialisation. It does not tell us how the system performs on messy review questions such as: “What is the state of evidence for intervention X, excluding weak observational studies, and how has the conclusion changed since 2021?” That would require study-quality assessment, domain-specific inclusion rules, conflict handling, and probably a human reviewer with coffee and scepticism.

The evaluation metrics are standard RAG-style measures: Faithfulness, Answer Relevance, Context Precision, and Context Recall. The authors also use 12 bootstrap resamples, each with 20 randomly selected questions split evenly between VectorRAG and GraphRAG, and report uncertainty estimates with a stated significance level of 0.05. The paper describes this as supporting statistical significance, with standard error at or below 0.10.

The important interpretation: the benchmark tests whether routing plus DPO improves retrieval-grounded answering on controlled graph and vector tasks. It does not establish autonomous literature synthesis across the real scientific workflow. That is not a criticism. It is the difference between a useful experiment and a procurement fairy tale.

The biggest gains are retrieval-quality gains, not prose-quality gains

The headline numbers are strong because they are concentrated in retrieval-sensitive metrics. Compared with the non-agentic baseline, the DPO-enhanced agentic system improves VS Context Recall by 0.63 and overall Context Precision by 0.56. In practice, that means the system retrieves more of the context needed for vector questions and ranks relevant context more effectively across the combined task set.

Additional improvements are more modest but still directionally useful: +0.24 in VS Faithfulness, +0.12 in VS Precision, +0.12 in KG Answer Relevance, +0.11 in overall Faithfulness, +0.05 in KG Context Recall, +0.04 in VS Answer Relevance, and +0.04 in overall Precision. The paper also reports slight decreases against the baseline in KG Precision (-0.04) and KG Faithfulness (-0.03), which it links to remaining difficulty in precise Cypher generation.

That last point deserves attention. The system’s graph side is not a solved problem. Graph retrieval is only as reliable as three things: the graph schema, the data ingested into that schema, and the translation from user intent into a valid query. If the natural-language-to-Cypher step misfires, the system can retrieve precisely the wrong thing. Precision is very comforting when aimed at the wrong target. So is a sniper rifle, but nobody calls it a governance framework.

A cleaner way to read the evidence is this:

The evidence therefore supports a mechanism claim more than a model claim. It is not “LLaMA plus Mistral beats baseline.” It is “a system that chooses between structured and semantic retrieval can retrieve and ground better than a system that flattens everything into one retrieval path.” That is the part operators should keep.

The uncertainty story is useful, but not yet a calibrated confidence product

The paper repeatedly emphasises uncertainty quantification. The concrete evidence provided is bootstrap-based evaluation: repeated resampling of benchmark questions to estimate variability in metrics. That is useful. It tells readers whether observed gains are likely to be stable under resampling of this test set.

But evaluation uncertainty is not the same as per-answer confidence calibration. A production research assistant needs to tell a user: “This answer is weak because only two documents support it,” or “The graph path failed to resolve this entity,” or “The vector context is relevant but contradictory,” or “This question requires tables we did not parse.” The paper gestures toward runtime uncertainty and low-confidence flagging, but the detailed experimental evidence is stronger for bootstrap uncertainty over benchmark results than for calibrated confidence attached to each answer.

That distinction matters in regulated or high-stakes domains. A dashboard showing confidence intervals over offline RAG metrics is helpful for model governance. It does not automatically make every generated answer safe to use. If an investment research platform, clinical evidence assistant, or legal knowledge tool adopts this architecture, it should separate two layers:

The paper provides a useful start on the first layer and a motivation for the others. Businesses should not collapse them into one comforting “confidence score.” Comforting scores are how dashboards become decorative furniture.

What this means for enterprise research systems

The immediate business relevance is not confined to academic literature review. Many organisations have exactly this problem under different names. Pharmaceutical teams search trial reports, regulatory filings, protocols, and publications. Law firms search case law, filings, opinions, and internal memos. Industrial R&D teams search patents, papers, test reports, and supplier documentation. Financial analysts search disclosures, transcripts, research notes, and market commentary.

In all these settings, knowledge has both structure and text. Structure tells you who, when, where, and how records connect. Text tells you what was argued, observed, disclosed, measured, or promised. A research system that does not distinguish these layers will either overuse semantic search or over-engineer a graph that cannot capture full meaning.

The paper suggests a more useful architecture:

1. Store explicit relationships as relationships, not chunks.
2. Store full text as retrievable evidence, not pseudo-structured metadata.
3. Route each question before retrieval, not after generation has already begun.
4. Use reranking to compress noisy vector results.
5. Use preference tuning to reward grounded answers.
6. Use uncertainty estimates and human oversight to prevent confident automation from becoming confident error.

For ROI, the most plausible gains are not “fewer researchers.” The near-term gains are lower search friction, faster evidence triage, more traceable answers, better reuse of existing corpora, and fewer hallucinated summaries. That is still valuable. It is just less cinematic than replacing a review team with a chatbot wearing a lab coat.

For technical leaders, the paper also implies that RAG evaluation should be segmented by question type. A single aggregate score can hide whether the system is good at relationship lookup but poor at evidence synthesis, or strong on full text but weak on citation traversal. The authors’ KG/VS split is a useful pattern: test the system by retrieval need, not by generic “question answering.”

The practical boundaries are where production work begins

The paper is careful enough to name several unresolved issues. These are not footnotes to be waved away after the demo. They are the implementation roadmap.

First, Cypher translation remains fragile. The GraphRAG path relies on few-shot prompting to convert natural-language questions into graph queries. Complex queries can be misinterpreted, and the authors propose fine-tuning on curated natural-language/Cypher pairs or using function-calling APIs. For enterprise deployment, this is not optional polish. If graph queries drive compliance or research conclusions, query validation and schema-aware testing are core controls.

Second, source coverage depends on available, parseable documents. The system downloads and chunks open-access full texts where available. Scanned or poorly structured PDFs require OCR, which the authors identify as future work. In corporate archives, scanned contracts, image-heavy reports, and badly exported PDFs are not edge cases. They are Tuesday.

Third, multimodal content is not yet handled. The authors explicitly note that synthetic benchmarks may not capture real scientific inquiries involving figures and tables. That is a serious boundary for scientific and financial research. Many critical findings live in tables, diagrams, plots, appendices, and supplementary files. A literature assistant that cannot reason over them may retrieve the prose around the answer while missing the answer itself.

Fourth, the benchmark is small and synthetic. Forty question-answer pairs are enough to demonstrate a mechanism, not enough to certify a system. The healthcare-MLLM search domain is also narrow. Performance needs validation on real corpora, real users, and real workflows where questions are underspecified, contradictory, and politically inconvenient.

Fifth, external APIs introduce variability. PubMed, arXiv, and Google Scholar availability, rate limits, metadata quality, and access rules can all affect ingestion. In production, data pipelines fail in ways that models politely do not mention. Monitoring ingestion health is part of model quality.

Sixth, latency matters. The authors report approximately two minutes per query on a consumer-grade machine and around ten seconds with suitable server infrastructure and GPU acceleration. Ten seconds may be acceptable for research synthesis. It is less acceptable for interactive exploration unless users understand what they are waiting for. The faster system is not just a model decision; it is an infrastructure decision.

The better mental model: cite before you write

The paper’s most useful message can be reduced to one operating principle: cite before you write.

Not literally in the final prose sequence, but architecturally. Before the generator speaks, the system should know what kind of evidence it needs. If the question is relational, traverse the graph. If the question is textual, retrieve and rerank passages. If the evidence is missing, say so. If the query translation is uncertain, flag it. If the answer requires a table or scanned figure that the system cannot parse, do not improvise. Improvisation is charming in jazz. In research automation, it is usually a liability with paragraph breaks.

This is why the paper’s mechanism-first interpretation is more useful than a plain summary. The performance numbers are encouraging, but the deeper lesson is architectural. Enterprises do not need more fluent research assistants that hide retrieval choices behind a smooth answer. They need systems that expose how evidence was found, why a retrieval path was chosen, and where the answer may fail.

Agentic RAG should therefore be judged less by whether it sounds autonomous and more by whether it makes the research workflow inspectable. The router is not a convenience feature. It is the control surface.

Conclusion: the agent is useful because it narrows the machine’s confusion

This paper does not prove that autonomous agents can replace literature reviewers. Good. That would be a rather ambitious claim for a 40-question synthetic benchmark, and ambition is not the same thing as evidence.

What it does show is more practical: a hybrid RAG system improves when it dynamically chooses between graph-based and vector-based retrieval, rather than forcing every question through a fixed retrieval path. The reported gains, especially the +0.63 improvement in VectorRAG Context Recall and +0.56 improvement in overall Context Precision, support the idea that retrieval routing can materially improve grounded answering. DPO adds another layer by nudging the generator toward evidence-based responses, though its effects vary by task type.

For business use, the value is not merely faster literature review. It is more governable research intelligence. The system’s design creates clearer boundaries between metadata relationships, full-text evidence, retrieval selection, generation, and uncertainty estimation. Those boundaries make the system easier to evaluate and easier to improve.

The next step is not to declare victory over systematic review. It is to test this architecture on messier corpora, richer document types, stricter domain protocols, and real users who ask questions no synthetic benchmark writer would be kind enough to preformat.

Until then, the sensible takeaway is this: before an AI system writes the answer, make it prove it knows where the answer should come from.

Cognaptus: Automate the Present, Incubate the Future.