Hive Minds and Hallucinations: A Smarter Way to Trust LLMs

TL;DR for operators

The paper is useful because it treats hallucination less like a mystical defect of large language models and more like an operational risk that can be routed, checked, scored, and sometimes refused.

Amer and Amer propose a proof-of-concept multi-agent architecture for SMS-based pharmacy prescription-renewal requests.¹ A customer might send a clean message like “1, unenroll”, or something messier: a renewal code, a complaint about medicine taste, a question about blood-pressure medication, and a polite thank-you bundled into one little administrative grenade.

The system does not simply pass the message to an LLM and hope the model behaves like a licensed pharmacist with good paperwork habits. It first uses deterministic parsing and fuzzy confidence scoring. Only when the message becomes ambiguous does it involve LLM agents, specifically Gemini and ChatGPT through LangChain4J. Then a validator checks whether the LLM output agrees with parser evidence, whether the LLM invented keywords, whether the two models disagree, and whether risky medication-related actions require confirmation.

That is the business lesson. The paper is not strong evidence that multi-agent systems “solve hallucination”. It is a narrow prototype with ten crafted SMS examples and one repeated 50-run test. What it offers is a practical control pattern: let deterministic systems handle what they can prove, let LLMs handle what they can interpret, and let validation agents decide what is safe enough to execute.

In other words: not “trust the hive mind”. More like “make the hive fill out a reconciliation report before touching the prescription queue.” Glamorous? No. Sensible? Annoyingly, yes.

The customer message is short; the operational risk is not

Customer-service automation often begins with a cheerful fiction: customers will express one intent at a time, use the right keywords, and avoid turning a two-character reply into a mini novella.

Actual customers have other plans.

In the pharmacy scenario used by the paper, customers receive automated SMS messages asking whether they want to renew or stop renewal for expiring prescriptions. Each medication is associated with a code. A customer may reply with several codes, combine renewal and stop instructions, add a complaint, ask a question, and include normal human padding such as “please” and “thank you”.

A traditional rules system can handle “1, unenroll”. It struggles when the same message also asks whether a medication should taste bad or whether a doctor needs to approve another renewal. An LLM can interpret the natural language, but it may also invent an instruction, omit a keyword, or over-read a casual phrase. In a pharmacy workflow, that is not a charming quirk. It is a liability wearing a chatbot costume.

The paper’s architecture is designed around this tension. The authors are not trying to make a single model smarter. They are decomposing the customer-service task into smaller roles, each with a different tolerance for uncertainty.

That is why the mechanism matters more than the headline.

The core mechanism: separate certainty, ambiguity, and action

The system’s central move is to split message handling into agents with distinct jobs.

At a high level, the flow is:

Incoming SMS
   ↓
Orchestration Agent
   ↓
Renewal Agent: deterministic parsing + fuzzy confidence
   ↓
Evaluator Agent: decide process, fail, or forward to LLM
   ↓
LLM Agents: Gemini and ChatGPT extract keywords, requests, complaints
   ↓
Validator Agent: compare, score, discard, retry, confirm, or escalate
   ↓
Router Agent
   ↓
Specialist agents: pharmacist, store management, scheduling, complaints

This looks more complicated than “send SMS to LLM”. That is the point.

A single LLM call collapses several different decisions into one opaque output: What did the customer mean? Which words matter? Which action should the business take? Is the action risky? Should a human intervene? The proposed system pulls those decisions apart.

The architecture makes a useful distinction between three kinds of work:

The design is not “LLMs plus agents”. That phrase has become almost impressively uninformative. The design is a control system around an LLM.

The Renewal Agent creates a baseline the LLM must answer to

The Renewal Agent is the first serious filter.

It caches known keyword pairs and regular expressions from a database, removes known politeness expressions, splits the SMS into words, and attempts to match each word against renewal or stop-renewal keywords. If every relevant word can be accounted for, the message can be processed directly.

The important detail is not just that the parser finds keywords. It also produces a fuzzy variable called degree of confidence. The labels are high, intermediate, and low, calculated from the percentage of words identified against the total message content. If the parser identifies 100% of the keywords, the high-confidence value is 1 and the other labels are zero.

That creates a baseline.

The LLM is no longer floating in interpretive space, free to be persuasive and wrong. It is being compared against a deterministic pass. The parser may be incomplete, especially in messy language, but what it does extract is treated as reliable. This matters because hallucination detection needs a reference point. Without one, a second model can disagree with the first model, but disagreement alone does not tell the system which one is wrong. Two confident interns are still interns.

The paper’s system uses the parser as the first audit trail.

Fuzzy logic turns uncertainty into routing, not drama

Once the Renewal Agent has scored the message, the Evaluator Agent decides what happens next.

If high confidence is 100%, the action can go straight to the pharmacy service endpoint: renew this medication, stop renewal for that one. No LLM is needed. No heroic reasoning. No expensive model call just to understand “1, unenroll”. Automation should not invoice the business for poetry.

If high confidence is below 100%, the message contains words or sentences the parser could not match. The Evaluator Agent then uses fuzzy rules to decide whether to forward the message to the LLM agent or fail the message and ask the customer to contact support.

The paper also introduces a Customer Relationship Agent. It calculates a fuzzy customer-importance variable from years since joining and total purchases in the previous 12 months. The Evaluator Agent combines that with the parser confidence score to decide whether the message deserves LLM processing.

This is operationally interesting, and slightly uncomfortable.

From a service-management point of view, customer importance is a practical triage signal. From a healthcare-adjacent point of view, using purchase history to influence automation handling deserves governance. The paper presents it as a routing variable, not as an ethical argument. In a real deployment, that choice would need policy review. Some queues can be value-based. Medication-related access and safety workflows are not always the ideal playground for “VIP gets smarter automation”.

Still, the architectural idea is useful: ambiguous cases should not all be treated equally. A production system could replace customer value with clinical risk, regulatory exposure, complaint severity, medication criticality, or some blend of all four.

The mechanism survives even if the variable should change.

The LLM agents interpret what rules cannot

When the Evaluator Agent forwards a message to the LLM layer, the system calls both Gemini and ChatGPT via LangChain4J. The LLM prompt uses a few-shot approach and asks for structured output.

The desired response contains arrays for:

• keywords used to renew medication;
• keywords used to stop medication renewal;
• complaints;
• requests;
• customer mood.

This output format is not a decoration. It is what makes validation possible.

Free-form LLM replies are hard to compare, hard to audit, and hard to connect to downstream systems. Structured extraction turns the model into one component in a workflow. It does not eliminate hallucination, but it gives the system handles for detecting it.

The paper’s broader point is that LLMs are valuable where language is genuinely messy: complaints, requests, informal phrasing, multi-intent SMS messages. They are less valuable where a deterministic system can already do the job. That is a useful design principle for any company building customer-service automation.

Use LLMs for ambiguity. Do not use them as a very expensive regular expression engine wearing a blazer.

The Validator Agent is the real hallucination-control layer

The Validator Agent is the most important part of the design.

It evaluates LLM output in two stages.

First, it checks keyword extraction. The Validator compares the keywords found by the Renewal Agent with those produced by the Gemini and ChatGPT agents. If the parser found a keyword that an LLM omitted, the LLM response is considered suspect. If the LLM detects a keyword that does not appear in the original SMS, that is treated as hallucination. If Gemini and ChatGPT produce non-discarded keyword outputs that still do not match, the system retries once. If disagreement remains, the customer is asked to contact support.

Second, it checks complaints and requests. The system asks Gemini to evaluate ChatGPT’s extracted complaints and requests, and asks ChatGPT to evaluate Gemini’s extraction. Each model scores the other response from 1 to 10. A score below 5 leads to discard. If both are discarded, the customer is sent to support. Otherwise, the higher-scoring response is selected.

This is not mathematically airtight. A model evaluating another model can be biased, lenient, or confidently wrong. But the operational design is still sensible: do not let a single generative pass become a business action.

The keyword stage is stronger than the complaint/request stage because it can compare LLM outputs against the original SMS and parser evidence. The complaint/request stage is weaker because it relies more heavily on model judgement. The paper recognises this indirectly by noting future work on more advanced techniques for improving complaint and request extraction.

For operators, this distinction matters. Validation quality depends on the reference signal. When there is a deterministic reference, validation is stronger. When validation is just “LLM judges LLM”, the architecture is better than nothing, but not yet a compliance shield. Lovely phrase, compliance shield. Usually means “the thing lawyers find holes in first”.

Risk scoring changes the action, not just the confidence label

The paper’s risk handling becomes most relevant when an LLM identifies additional keywords beyond the Renewal Agent.

If the extra keyword relates to a stop-renewal request, the Validator Agent evaluates risk using fuzzy rules. The system considers factors such as medication type, how long the patient has been taking the medication, and whether it is associated with a chronic or long-term disease. It then calculates a crisp high-risk value using centre-of-gravity defuzzification. If the value exceeds a configured threshold, the system sends a confirmation SMS to the patient. Otherwise, it processes the keyword through the pharmacy endpoint.

The important business lesson is not the specific fuzzy formula. It is the coupling between interpretation and consequence.

A low-risk inferred action may be executed. A high-risk inferred action requires confirmation. A disagreement may trigger retry or escalation. A failed extraction may produce a support handoff. The same uncertainty score does not always produce the same operational response; it depends on what could happen next.

That is how AI risk controls should behave. A hallucinated store-hours answer and a hallucinated medication stop request do not belong in the same risk bucket. The former creates irritation. The latter could create harm.

Specialist agents keep the workflow from becoming one giant chatbot

After validation, complaints and requests go to a Router Agent. Expert agents register their areas of expertise, and the Router chooses the right downstream handler. The paper defines four: pharmacist, store management, scheduling, and complaint department.

These agents are not all the same kind of AI.

The Pharmacist Agent forwards questions and requests to a pharmacist. The complaint department agent forwards complaints to customer support. The store management agent is a RAG agent for store locations and operating hours. The scheduling agent uses an LLM augmented with an appointment tool to interpret patient constraints and search for vaccination appointment slots.

This is a good architectural instinct. “Agent” should not mean “LLM everywhere”. Sometimes the right expert agent is a database lookup. Sometimes it is a tool-using model. Sometimes it is a human. The useful design question is not “How many agents can we add?” but “Which component should own this decision?”

A multi-agent architecture becomes valuable when specialization reduces error. It becomes theatre when it just multiplies model calls and calls the invoice intelligence.

The evidence is a prototype check, not a reliability benchmark

The paper’s evaluation is small and should be read accordingly.

The authors test the system on ten sample SMS messages placed in the event hub. These messages vary in structure and tone, including direct renewal commands, polite wording, stop requests, a flu-vaccine scheduling request, and messages containing additional questions or complaints.

They report that both the Renewal Agent and LLM agents successfully extracted relevant keywords, except in cases where keywords appeared inside more complex or multi-intent sentences. Two examples are especially informative.

In one message, the customer says: “Enroll. I want also to renew my blood pressure medication.” The Renewal Agent extracts “enroll” but does not treat “renew” as a false hit requiring stop-renewal handling. Since the word is not related to a stop keyword, the system treats it as low risk and sends it to the pharmacy for processing.

In another message, “Please stop sending me text messages”, the Renewal Agent extracts no keyword, while the LLM agents extract “stop”. The system’s next action depends on the medication associated with the keyword: it may process the request or send a confirmation SMS.

The authors also repeat the first, complex SMS question 50 times. In one run, one LLM response incorrectly adds an extra “renew” keyword that was not present in the original SMS text. The Validator Agent detects this and discards that response, using the other LLM response instead.

Here is the cleanest way to interpret the evidence:

The evidence is encouraging in the narrow sense: the designed control layer catches a real hallucinated keyword in the repeated test. But the paper itself states that a comprehensive assessment of success rate would require production deployment and evaluation, which is outside the study’s scope.

That boundary should not be treated as a footnote. It is the difference between “promising architecture” and “ready operating model”.

The business value is controlled automation, not blind replacement

For businesses, the paper’s value is not a new customer-service chatbot. It is a pattern for deciding where automation can safely act.

The most relevant use cases are customer operations where messages are short, semi-structured, and action-bearing:

• prescription renewals;
• appointment scheduling;
• insurance claim status requests;
• banking service instructions;
• logistics delivery changes;
• telecom support messages;
• membership cancellation or renewal flows.

These workflows share a common structure. Some messages are trivial and should be automated cheaply. Some are ambiguous and need language interpretation. Some trigger risky business actions. Some should be escalated immediately because the cost of a wrong action is high.

The paper suggests a practical automation stack:

The ROI pathway is also fairly clear. A company could reduce human handling of simple messages, reserve LLM costs for ambiguous cases, and reduce the chance that a hallucinated interpretation directly triggers a harmful action. That is a more credible business case than the usual “AI will transform customer experience” mist machine.

The uncertainty is equally clear. The paper does not measure cost savings, latency, human deflection rate, customer satisfaction, or production failure rates. Those would need a deployment study with real messages, realistic traffic, edge cases, monitoring, and incident review.

The awkward governance question: who gets escalated?

One design choice deserves more scrutiny than the paper gives it: the use of customer importance in routing.

The Customer Relationship Agent computes customer importance based on tenure and purchase value. The Evaluator Agent can use this variable when deciding whether an ambiguous message should be forwarded to the LLM or failed into a support instruction.

In ordinary retail, that might be defensible. High-value customers often receive richer service. In healthcare-related workflows, this becomes delicate. If automation quality, escalation paths, or message interpretation differ by customer commercial value, the business should be very clear about what is being optimised.

A safer production design would separate commercial prioritisation from safety prioritisation. For example:

• medication criticality should affect confirmation requirements;
• chronic-condition relevance should affect escalation;
• customer vulnerability or access needs may affect support routing;
• commercial value should not quietly determine whether a patient gets better interpretation of a medication-related instruction.

This is not a fatal flaw in the prototype. It is a reminder that multi-agent systems do not remove policy decisions. They encode them. Sometimes in YAML. Very modern. Very easy to overlook.

Where the architecture is strongest

The system is strongest where three conditions hold.

First, there is a deterministic subset of the task. Renewal and stop-renewal keywords are a good fit because the system can build a parser around known expressions and codes. This creates an audit baseline.

Second, the ambiguous part is linguistically messy but operationally separable. Complaints, requests, appointment preferences, and store-hours questions can be extracted and routed without immediately changing medication status.

Third, the business can define risk-sensitive actions. A stop-renewal request for a long-term medication can trigger confirmation. A store-hours question can go through RAG. A complaint about taste can go to a pharmacist. The architecture becomes more valuable when each output class has a different consequence.

This is why the paper’s mechanism-first framing matters. The result is not “two LLMs are better than one”. The result is “a narrow deterministic baseline plus model cross-checking plus consequence-aware routing can make LLM interpretation safer in a bounded workflow.”

That is less exciting. It is also much more likely to survive contact with operations.

Where the prototype is still thin

The main limitation is sample size. Ten SMS messages cannot represent the messiness of real customer language. Real messages include typos, multilingual phrasing, abbreviations, sarcasm, contradictory instructions, family members texting on behalf of patients, outdated codes, duplicate messages, and customers who write as though punctuation personally betrayed them.

The repeated 50-run test is useful but narrow. It repeats one complex message and observes one hallucinated extra keyword from one LLM response. That demonstrates the Validator Agent can catch that kind of error. It does not estimate the probability of hallucination, false discard, false acceptance, latency impact, or model disagreement under broader conditions.

The validation strategy is also uneven. Keyword validation benefits from parser evidence and original-text comparison. Complaint and request validation relies more on model-to-model scoring. That can help, but it is weaker than validation against an external source, human-labelled examples, or domain rules.

Finally, the architecture may introduce operational complexity. Event hubs, dynamic event processors, multiple agents, tracking agents, validators, routers, and specialist agents all need monitoring. A multi-agent system can reduce one class of model risk while adding orchestration risk. Messages can fail because the LLM hallucinated, because the validator was too strict, because the routing prompt misclassified the request, because the appointment tool returned a strange slot, or because the event pipeline quietly dropped a message. Distributed systems: because apparently one failure mode was not enough.

What Cognaptus would test before production

Before taking this pattern into a live regulated workflow, the next evaluation should answer five questions.

The production benchmark should also classify tests by purpose: main evidence for extraction accuracy, ablation tests without the parser or without dual-LLM validation, robustness tests across message styles and languages, and implementation tests under realistic traffic.

Without that, the prototype remains a good architectural sketch rather than a proven operating system.

The useful idea is not the hive; it is the audit trail

The paper’s title emphasises multi-agent architecture, and that is fair. But the deeper lesson is about auditability.

The system creates checkpoints. The Renewal Agent records what it can prove. The Evaluator Agent decides whether ambiguity deserves more processing. The LLM agents interpret what the parser cannot. The Validator Agent compares, rejects, retries, or escalates. The Router Agent sends the surviving request to a specialist.

That is how trust should be built in LLM workflows: not by assuming the model has become wise, but by narrowing what it is allowed to decide.

The paper does not prove that multi-agent systems eliminate hallucination. It shows a small, concrete way to make hallucination visible enough to act on. In customer operations, that may be the more valuable achievement.

The future of trustworthy LLM deployment will probably not look like one giant model answering everything. It will look like a mildly bureaucratic swarm of parsers, scorers, validators, tools, and humans, each doing a limited job and leaving enough evidence behind for someone to ask, “Why did the system do that?”

Not glamorous. But then again, neither is not accidentally stopping someone’s medication. Standards must be maintained.

Cognaptus: Automate the Present, Incubate the Future.