HyFedRAG: Caching Privacy into Federated RAG

Hospital search is rarely a search problem in the clean, consumer-internet sense. The useful information is not sitting in one tidy index, wearing a name badge, waiting to be embedded. It is scattered across clinical notes, relational databases, knowledge graphs, departmental systems, hospital networks, and legal boundaries. Naturally, this is where people decide to add a large language model and call it “modernisation.” Brave.

The HyFedRAG paper tackles this mess with a useful change of perspective: the question is not merely how to build a better RAG system, but how to decide where each part of RAG should happen when the data cannot safely move.¹ Its answer is an edge-cloud federated architecture for heterogeneous healthcare data. Text retrieval happens locally. SQL retrieval happens locally. Knowledge-graph retrieval happens locally. Local LLMs produce privacy-aware summaries. A central model receives only those de-identified summaries and fuses them into a global answer.

That placement choice is the paper’s real contribution. The retrieval numbers matter. The privacy evaluation matters. The cache hit rates matter. But the important business lesson is architectural: in regulated environments, the system boundary is not the model boundary. It is the data boundary.

HyFedRAG starts by refusing the fantasy of one clean medical index

Standard RAG likes a comforting assumption: collect documents, chunk them, embed them, retrieve relevant chunks, pass them to a model, generate an answer. This is perfectly pleasant until the documents are not just documents, the owners are not the same organisation, the data is protected by law, and some records are structured enough to be useful but too rigid to behave like text.

HyFedRAG frames the problem as similar-patient retrieval in a federated healthcare setting. Given a clinical case, the system must find similar patients across multiple local clients. Each client may hold a different modality: unstructured text, structured SQL records, or semi-structured knowledge graphs. The system should retrieve useful evidence without centralising raw patient data.

That “without centralising” phrase is doing a lot of work. A centralised RAG pipeline would normally pull records into a common cloud-side retrieval layer. HyFedRAG does the opposite. It pushes retrieval and preliminary summarisation to the edge, then lets the cloud work only on sanitised intermediate representations.

The mechanism is easiest to understand as a three-stage handoff:

This is not “privacy because we used federated in the name.” It is privacy by routing: raw data stays where it is, and the central layer receives a narrower object.

That narrower object is not risk-free. A summary can still leak. A de-identified sentence can still carry quasi-identifiers. A rare clinical combination can still point back to a person if the surrounding institution is small enough. The paper’s strongest practical idea is not that privacy is solved; it is that privacy becomes a controllable stage in the pipeline rather than an afterthought stapled to the output.

The edge layer is three retrievers wearing one uniform

HyFedRAG supports three local retrieval paths: text, SQL, and knowledge graph. They are not treated as interchangeable formats poured into the same embedding bucket. That is good. One of the more persistent enterprise mistakes in RAG is assuming every knowledge asset becomes equally useful once converted into text. The spreadsheet begs to differ.

For text, HyFedRAG combines sparse and dense retrieval. The sparse side uses TF-IDF and cosine similarity; the dense side uses embeddings and FAISS; a reranker then balances lexical matching with semantic relevance. This matters in medicine because a correct match may require both exact terminology and semantic flexibility. Exact disease names, drug names, and clinical terms are valuable. So are paraphrases, synonyms, and long-tail descriptions that do not share surface vocabulary.

For knowledge graphs, the system first extracts medical entities from the query, tries exact matching in Neo4j, then falls back to semantic matching where exact matching fails. It retrieves relation paths connected to patient nodes and reranks candidate statements before selecting patient-level summaries. In other words, the graph path is not just “embed the graph and hope.” It uses entity matching, relation traversal, and semantic reranking as separate steps.

For SQL, the system uses entity extraction, MySQL full-text search in Boolean and natural-language modes, application-level heuristics such as exact match and phrase similarity, and then a cross-encoder reranker. This is not glamorous, but it is realistic. Many enterprise systems do not need one spiritually enlightened foundation model. They need several stubborn retrieval adapters that can talk to existing data stores without requiring a total migration first.

The paper’s architecture therefore makes a sensible bet: heterogeneity should be handled locally, where the data format is still meaningful. The central server should not have to know whether a piece of evidence began life as a table row, a graph edge, or a paragraph. By the time it sees anything, the local client has already translated the modality into a privacy-conscious summary.

The privacy layer is useful, but narrower than the word “compliant” suggests

The privacy mechanism is built around local summary generation. Each client retrieves relevant material, runs a local LLM, and applies privacy tooling before sending summaries upstream. The paper discusses three tools: Presidio for PII detection and anonymisation, Eraser4RAG for context-aware removal of non-essential sensitive spans, and TenSEAL for encrypted tensor operations.

The conceptual stack is sensible:

The experimental evidence, however, should be read carefully. The paper’s implementation section says anonymisation is performed using Presidio. The privacy evaluation uses GPT-4o through DeepEval’s GEval-style scoring to compare outputs before and after the HyFedRAG privacy mechanism, using local LLM outputs from models such as Llama-3.1-8B-Instruct and Gemma-2-9B-IT. The reported result is that privacy scores improve after protection, while readability and information integrity are not noticeably damaged.

That is useful evidence, but it is not the same as a legal compliance proof. GEval privacy scoring is an evaluator-based assessment, not a formal privacy guarantee. Presidio masking is helpful, but it does not eliminate all re-identification risk. TenSEAL support is architecturally important, but the paper does not establish a live hospital-grade cryptographic deployment across all modalities. Eraser4RAG is positioned as part of the privacy toolkit, but the paper’s reported privacy evaluation appears centred on Presidio-based masking.

This distinction matters because “federated RAG” is the kind of phrase that can make executives and vendors simultaneously overconfident. The correct reading is more disciplined: HyFedRAG shows a promising privacy-aware architecture and reports improved privacy scores under its evaluation setup. It does not prove that an organisation can now bypass governance, clinical data-sharing agreements, threat modelling, audit logging, or regulator-facing validation. Unfortunately, lawyers remain undefeated.

Caching is not a side feature; it is how federated RAG avoids becoming slow theatre

Federated systems often pay a performance tax. Data stays distributed, so orchestration becomes more expensive. Local retrieval, local summarisation, network coordination, cloud fusion, and repeated LLM calls can turn a beautiful privacy architecture into a waiting-room experience. Healthcare already has enough of those.

HyFedRAG addresses this with a three-tier cache in the middleware layer. The tiers are described slightly differently across the paper’s architecture and analysis sections, but the operational idea is consistent: avoid repeating expensive work at multiple points in the pipeline.

The cache experiment is best understood as an ablation-style system efficiency test, not as primary retrieval evidence. The authors simulate clinical retrieval behaviour using 100 warm-up requests and 500 test requests generated through a random-walk process over a document-entity association graph. They then measure hit and miss rates across the three cache layers for text-only RAG, SQL, and knowledge-graph retrieval.

The reported pattern is strong. The first tier reaches hit rates around 45–48% depending on modality. The middle tier intercepts another 15–17% of requests. The bottom tier adds roughly 21–23%. Together, the cumulative hit rate rises above 84%, with miss rates around 14–16%. The paper links this to an approximate 80% reduction in average access latency.

There are two useful interpretations here. The obvious one is cost and speed: cache repeated work, reduce latency, reduce redundant computation. The less obvious one is behavioural. Clinical and enterprise search sessions are rarely independent single-shot queries. A user asks about a condition, then a subtype, then a treatment, then a similar case, then a lab marker. Query locality exists. HyFedRAG’s neighbour prefetching tries to exploit that structure.

That makes the caching design more interesting than a simple “store popular answers” trick. It is not only caching outputs; it is caching around likely retrieval paths. In enterprise terms, that is the difference between a filing cabinet and an assistant who knows the next drawer you are probably going to open.

The headline retrieval result is strong, but the format gap is the more instructive result

The main retrieval result compares HyFedRAG’s text setting against existing PMC-Patients baselines. The paper reports that HyFedRAG(text) reaches 39.63% MRR, 7.48% P@10, and 41.33% nDCG@10. The best listed baseline, RRF, reaches 27.76% MRR, 6.96% P@10, and 24.12% nDCG@10. In the table’s terms, HyFedRAG improves over the strongest baseline by 11.87 points in MRR, 0.52 points in P@10, and 17.21 points in nDCG@10.

The business reading should not be “HyFedRAG wins, buy confetti.” The more useful reading is that hybrid retrieval plus reranking is especially valuable when the source material remains text-rich. Text preserves clinical context: symptom descriptions, temporal patterns, narrative qualifiers, and the messy language of case reports.

The cross-format results are more sobering:

This is one of the paper’s most useful results for practitioners. Text performs best. SQL falls substantially. Knowledge graph retrieval falls further. The authors attribute this to semantic information loss during structured-format construction.

That finding should make enterprise data teams uncomfortable in a productive way. Structured data is not automatically superior for AI retrieval. A table can be precise and still semantically thin. A graph can encode relationships and still miss the context that made those relationships clinically meaningful. If the pipeline strips away the narrative that explains why an entity matters, the retrieval layer may become cleaner and less useful at the same time. A familiar bargain, usually discovered after the migration budget has already been approved.

For business deployment, the implication is not “avoid SQL and graphs.” That would be silly. The implication is that structured retrieval needs semantic augmentation. If a hospital, insurer, or research network wants to use federated RAG across structured stores, it may need richer schema descriptions, field-level text expansion, ontology-aware retrieval, graph-context summarisation, and careful preservation of narrative notes. Otherwise, the system may be privacy-preserving, federated, and impressively under-informed.

The fusion-weight analysis says retrieval still needs old-fashioned tuning

The paper also analyses the fusion weight in the text retrieval module. This analysis is best read as a sensitivity test for the sparse-versus-semantic balance. Performance improves as the fusion weight rises toward a reported sweet spot around 0.8, then degrades when pushed to 1.0. The authors interpret this as evidence that exact token overlap helps, but pure lexical matching loses relevant cases described in different language.

This is not a shocking result. It is, however, a useful reminder. Medical retrieval sits between two failure modes. If the system leans too much on semantic embeddings, it may retrieve plausible but clinically imprecise material. If it leans too much on exact terms, it misses long-tail cases where the same condition is described indirectly or idiosyncratically.

The important operational point is that HyFedRAG has knobs. The fusion weight is not just a research hyperparameter; in a deployed system it becomes part of retrieval policy. A rare-disease search tool, an adverse-event review tool, and a billing-code audit tool may not want the same lexical-semantic balance. The same architecture could support all three, but not with one universal setting blessed by a dashboard.

What each experiment actually supports

The paper combines retrieval metrics, privacy scoring, modality comparison, fusion-weight analysis, and cache simulation. These should not be blended into one vague “it works” conclusion. They answer different questions.

This separation matters because buyers and builders tend to over-package research results. A retrieval benchmark becomes a product claim. A privacy score becomes a compliance claim. A cache simulation becomes an infrastructure forecast. Each may be directionally useful. None should be promoted into a guarantee without additional validation.

The business value is not “AI over hospital data”; it is controlled cross-boundary reuse

The commercial appeal of HyFedRAG is easy to see. Hospitals want to search across cases without exposing raw records. Insurers want evidence retrieval without merging every sensitive dataset. Clinical research networks want rare-patient discovery across institutions. Pharmaceutical safety teams want signal investigation across fragmented sources. Large enterprise groups outside healthcare have similar problems: subsidiaries, jurisdictions, customer databases, legal walls, and incompatible systems.

HyFedRAG maps to that world because it treats cross-boundary AI as a systems problem. The direct paper result is a federated RAG design evaluated on PMC-Patients-derived text, SQL, and KG formats, with privacy-aware local summarisation and simulated cache gains. The Cognaptus inference is broader: organisations should stop thinking of RAG as one central index and start thinking of it as a governed choreography among local retrievers, privacy filters, caches, and central synthesis.

The return-on-investment pathway is plausible but conditional:

The key word is “conditional.” The paper’s business value becomes real only if the organisation can manage local deployment, model governance, data permissions, audit trails, cache invalidation, and evaluation against actual workflows. HyFedRAG gives a pattern. It does not ship a hospital IT department in a box, which is probably for the best.

Where the architecture would need hardening before production

The paper’s limitations are not fatal, but they are material.

First, the evaluation is based on PMC-Patients-derived data, with SQL and KG versions constructed for the experiment. This is useful for controlled comparison, but live hospital records are noisier, more inconsistent, and more politically defended than benchmark datasets. A real deployment would need site-level evaluation, not just aggregate retrieval metrics.

Second, privacy is assessed using evaluator-based scoring rather than formal guarantees. The paper’s future-work note mentions tighter integration of differential privacy. That is the right direction. For production, organisations would also need re-identification testing, red-team prompts, data minimisation policies, access controls, logs, and reviewable redaction behaviour.

Third, the cache experiment is simulated. It is intelligently designed around query locality, but production caching introduces unpleasant details: stale summaries, changing patient records, updated consent status, revoked access, audit requirements, and different invalidation rules by jurisdiction. Caching sensitive summaries can improve latency, but it also creates a new object that must be governed. The cache is not just an optimisation layer; it becomes part of the privacy surface.

Fourth, the modality gap is large. Text retrieval performs far better than KG retrieval in the reported setup. If an enterprise’s most important data is heavily structured, HyFedRAG’s architecture remains relevant, but the retrieval adapters will need serious domain-specific work. Converting everything into a graph and expecting magic is still not a strategy. It is a graph-shaped wish.

Finally, the paper’s architecture assumes local compute capacity for retrieval, reranking, summarisation, and privacy operations. The experiments run local models on two NVIDIA RTX A6000 48GB GPUs. That does not invalidate the approach, but it affects deployment economics. Edge-cloud collaboration still requires a capable edge.

The useful lesson: move less data, but move better summaries

HyFedRAG is valuable because it refuses two bad options. It does not centralise raw healthcare data just to make RAG convenient. It also does not pretend that local silos can answer cross-institution questions without coordination. Instead, it inserts a translation layer: retrieve locally, summarise privately, cache intelligently, fuse centrally.

That is the mechanism-first lesson for enterprises. The next generation of useful RAG systems in regulated environments will not be defined only by larger context windows or prettier chat interfaces. They will be defined by where retrieval happens, what representation crosses the boundary, how privacy is checked, and which intermediate objects are safe enough to cache.

The paper’s results are promising: strong text retrieval on PMC-Patients, improved evaluator-rated privacy after protection, clear evidence that format conversion can damage retrieval quality, and simulated cache behaviour that suggests major latency gains. The boundaries are equally clear: not live hospital proof, not formal compliance, not a universal fix for structured data, and not a free lunch on infrastructure.

Still, the design points in the right direction. If centralised RAG was the first draft of enterprise retrieval, HyFedRAG is closer to the adult version: less dramatic, more governed, and aware that the data has rights, owners, formats, and lawyers.

Cognaptus: Automate the Present, Incubate the Future.