The Health Bot Failed Before It Answered

TL;DR for operators

The paper is useful because it refuses to treat the healthcare chatbot as a lonely little model floating in a lab. It studies AI healthcare chatbot apps as an information infrastructure: an arrangement of access rules, interfaces, subscriptions, support channels, user expectations, and data practices wrapped around conversational software.¹ That framing matters because users often encounter the failure before they ever get to the allegedly intelligent part.

The researchers collected 264,310 reviews across 59 AI healthcare chatbot apps, filtered them to 213,182 English-language reviews, then used a sentiment classifier to isolate 15,090 negative reviews. They applied LDA topic modeling as a discovery aid, selected a 10-topic solution, and then grouped those topics through interpretive analysis into three higher-level concern categories: access barriers and service unreliability; user experience and AI interaction quality; and billing, customer support, and trust.

For product leaders, the headline is not “healthcare chatbots sometimes disappoint users.” Truly, hold the front page. The sharper point is that operational breakdowns become trust breakdowns. Login failures, surprise paywalls, trial conversions, vague cancellation flows, weak support, generic chatbot responses, and unclear data practices are not peripheral product dirt. In health-sensitive contexts, they are part of the care-facing system, even when the company would prefer to call them “growth,” “monetization,” or “standard SaaS plumbing.”

The most common concern category is interaction quality, with 9,118 reviews, or 60.4% of the negative corpus. But the most severe average ratings appear in billing and customer support: 2,775 reviews, mean rating 1.548, median 1. Access barriers also score very badly: 3,197 reviews, mean rating 1.713, median 1. Interaction quality has a higher mean rating of 2.875 and median 3, which suggests a more mixed state: users can be frustrated by shallow or generic interactions while still seeing some partial value. That distinction matters for triage. Some failures end the relationship immediately; others reveal where the product is still being given a chance.

Privacy, security, and data-handling complaints are rare in the negative-review corpus: 118 reviews are explicitly flagged by the authors’ keyword method. But those reviews have a much lower average rating than non-flagged reviews: 1.23 versus 2.39, with a reported $t \approx 16.67$ and $p < .001$. The business interpretation is not that users never care about privacy. It is that explicit privacy language often appears when a broader trust relationship is already deteriorating. Privacy is not only a compliance page. It is also a late-stage smoke alarm.

The evidence should be used as a failure taxonomy, not as a market-size estimate. It does not tell us what percentage of all users are harmed, what clinical outcomes changed, whether one demographic group is more exposed, or whether the complaints reflect representative global experience. It does tell us where users say these systems break in everyday use. That is already enough to make several product roadmaps look less mature than their investor decks.

The mistake is looking only for bad medical answers

The easy story about healthcare chatbots is clinical risk: the bot gives wrong advice, misses a crisis, hallucinates a treatment, or behaves like a confident intern with no supervisor and unlimited server time. That risk is real. It is also not the only place where the user relationship breaks.

This paper’s useful move is to shift attention from the answer box to the system around the answer box. A healthcare chatbot is not experienced as a model architecture. It is experienced as an app that asks questions, demands registration, gates features behind payment, sends or fails to send messages, charges subscriptions, responds through support channels, logs sensitive information, and sometimes produces what the user recognizes as a canned script wearing an AI costume.

That is why the paper’s “information infrastructure” framing is more than academic wallpaper. Infrastructure becomes visible when it breaks. The chatbot’s intelligence may be the advertised object, but the user’s actual experience is routed through access, reliability, interaction, support, billing, and data governance. A model can be impressive in a benchmark and still live inside an app that users experience as unreliable, evasive, or financially slippery. This is the glamorous future of digital health: sometimes the first clinical-grade intervention is a working login screen.

The paper’s category-based results make this clear. The three categories are not three synonyms for “bad UX.” They describe different failure locations in the user journey:

This is the article’s organizing point: the healthcare chatbot does not fail in one place. It fails as a chain. Product leaders should stop asking whether “the AI” works as though the user ever meets the AI in isolation.

What the researchers actually measured

The study begins with a broad app-store collection. The authors identified AI-enabled healthcare chatbot apps from Google Play and the Apple App Store using keyword searches such as AI health, AI healthcare, and healthcare chatbot. Apps had to be health-focused and provide AI-based conversational or chatbot functionality, including rule-based, proprietary machine-learning, or LLM-based components. Two researchers independently screened apps using public store materials, with Cohen’s $\kappa = 0.6773$, then resolved disagreements. The final app set contains 38 Android apps and 21 iOS apps, including 18 available on both platforms.

The review corpus is large: 264,310 reviews from June 2011 to October 2025. After English-language filtering, the authors retained 213,182 reviews. They then used TextBlob sentiment classification to select 15,090 reviews labeled negative. That filtered corpus is the analytical base. It is not “all users.” It is users writing in English, in public app-store settings, whose reviews were classified as negative.

The topic modeling is best read as structured exploration, not mechanical truth extraction. The authors trained LDA models over different topic counts, selected a 10-topic solution using coherence score and interpretability, then examined high-probability terms and representative reviews for each topic. They explicitly describe the topic model as a heuristic guide rather than a category machine. Two authors then grouped the ten topics into three broader categories.

That matters because the study’s strongest claim is not “LDA discovered the essence of chatbot suffering.” The stronger and more defensible claim is that computational topic modeling helped organize a large complaint corpus into recurring patterns, which were then interpreted by researchers. In other words, the paper is a map of negative user experience, not a randomized audit of product safety.

A small methods translation helps:

That is the right level of trust to place in the study: high enough to use it for operational diagnosis, not high enough to pretend it estimates clinical harm or market-wide dissatisfaction rates.

Access failures are not onboarding friction when the user came for support

The first category, access barriers and service unreliability, contains 3,197 reviews. Its mean rating is 1.713 with a median of 1. This is not polite disappointment. It is the sound of a user relationship being dropped on the pavement before the product has the chance to do anything clever.

The category contains three topics. The largest is access barriers, with 2,076 reviews and a mean rating of 1.58. The complaints include apps that appear free until core functionality is locked behind payment, failed logins, crashes, unavailable services, and app behavior that prevents use. The second topic, performance and mental health issues, is much smaller at 230 reviews but more sensitive: users connect technical failure or irrelevant chatbot output with worsened emotional states. The third, UX dissatisfaction, includes 891 reviews that express broad, intense negativity about the service.

The business lesson is simple and therefore frequently ignored. In digital health, the “front door” is not a neutral funnel. A login failure, a surprise paywall, or an app that stops responding is not just conversion leakage. It changes the user’s interpretation of whether the service can be depended on when the context is personal, emotional, or urgent.

This is where many health AI strategies quietly import consumer-app assumptions that do not survive contact with the category. For a casual entertainment app, a bad onboarding flow wastes time. For a mental-health support app, the same obstruction may arrive when the user is actively seeking reassurance, guidance, or relief. The paper cannot prove harm from those moments, and it does not claim to. But it does show that users describe access and reliability failures as central breakdowns, not decorative annoyances.

The operational implication is that access reliability should be governed like core product safety infrastructure. That does not mean every app must be free. It means pricing, trial rules, registration demands, and feature gates must be visible before emotional commitment has already been solicited. The user should not have to complete a vulnerability-shaped onboarding journey only to discover the useful part was behind a subscription curtain. Very innovative, yes. Also corrosive.

Interaction quality is where “AI” becomes a promise users can audit

The second category, user experience and AI interaction quality, is the largest: 9,118 reviews, or 60.4% of the negative corpus. Its mean rating is 2.875 and median is 3. That higher rating is not evidence that the category is harmless. It means dissatisfaction here is more varied. Some users are angry enough to leave one-star reviews. Others are still partially engaged, disappointed, or conflicted.

This category includes five topics: interface and design issues; perceived uselessness; poor emotional support; lack of responsiveness and personalization; and outdated or low-quality AI agents. The largest topic in the entire dataset is poor emotional support, with 3,435 reviews and a mean rating of 4.059. That number is worth pausing over because it is counterintuitive inside a negative-review study. Users can mark a review as negative by sentiment while still giving a relatively high star rating. The likely interpretation is that some users found pieces of value, but not enough emotional relevance, consistency, or clarity.

The low-rated interaction topics tell a sharper story. Perceived uselessness has 1,261 reviews and a mean rating of 1.621. Outdated and low AI agent quality has 2,226 reviews and a mean of 2.216. Lack of responsiveness and personalization has 1,759 reviews and a mean of 2.454. These are not merely complaints about color palettes or button placement. Users are evaluating whether the system understands them, adapts to them, and provides information or support that feels specific enough to matter.

This is where “AI” becomes a liability if the product behaves like a script. Once a service markets itself as AI-powered, users bring expectations shaped by modern conversational systems. They expect context sensitivity, personalization, responsiveness, and some visible ability to handle nuance. If they instead receive trigger-word responses, generic advice, or decision-tree behavior, the disappointment is not only functional. It is interpretive. The user concludes that the product is pretending.

For operators, this category should not be managed as a generic UX backlog. It needs capability governance. That means identifying which user claims the chatbot is expected to handle, which contexts require escalation, which advice must be personalized, and which interactions should never be simulated if the system cannot responsibly support them. A “friendly” bot that repeats shallow reassurance may be better than nothing in some contexts and actively irritating in others. The paper’s review evidence cannot adjudicate clinical quality, but it does show that users notice when the interaction is thin.

The rating distribution also implies different repair economics. Access and billing failures often produce immediate rejection. Interaction failures may still leave room for recovery because some users continue engaging despite dissatisfaction. That makes this category a candidate for iteration: better personalization, clearer scope boundaries, better escalation, and more honest capability claims. The point is not to make the bot sound more empathetic. It is to make the service less counterfeit.

Billing and support are not back-office details; they are trust machinery

The third category is smaller by volume but harsher by rating. Billing, customer support, and trust contains 2,775 reviews, or 18.4% of the negative corpus. Its mean rating is 1.548, the lowest among the three categories, and its median is 1. The 75th percentile is also 1, meaning dissatisfaction is heavily concentrated at the bottom.

The category contains two topics. Charges and refund concerns account for 2,086 reviews, with a mean rating of 1.555. Customer support and billing account for 689 reviews, with a mean rating of 1.527. Users report unexpected charges, unclear trial renewals, cancellation difficulty, refund refusals, weak customer support, and paid features that do not meet expectations.

It would be tempting to dismiss this as ordinary app-store billing drama. That would be convenient, and therefore wrong. In healthcare-adjacent products, billing design is part of trust design. The user is not buying a puzzle game skin. They may be seeking help for anxiety, nutrition, symptoms, mood, or self-management. When the payment system feels opaque or adversarial, the user’s suspicion does not stay neatly inside the finance workflow. It spreads to the service itself.

This is one of the paper’s most important business implications. The trust stack is wider than the model stack. Billing, support, cancellation, refund handling, and value communication are not customer-success afterthoughts. They shape whether users interpret the AI as a supportive service or as a subscription trap with a chatbot mascot.

The equity angle is also practical, not ornamental. The authors note that users may turn to these tools as lower-cost alternatives to formal care. If access is mediated by unclear subscriptions, surprise charges, and poor redress, the system can reproduce a familiar digital-health pattern: advertised accessibility, operational exclusion. The app is technically available. The useful service is not reliably reachable, understandable, or contestable.

For companies, this suggests a brutally concrete checklist. Can a user understand the price before onboarding? Can they cancel without theater? Are trial conversions explicit? Is refund policy written for humans? Does support respond when money has already changed hands? Are paid AI “coaches” or plans clearly distinguished from licensed clinical expertise where relevant? These questions are not beneath the strategy team. They are the strategy team’s homework, assuming the strategy is not “extract and apologize later.”

Privacy complaints are rare, but they mark a deeper trust rupture

The paper adds a complementary analysis of security, privacy, and data-handling concerns, which it abbreviates as SPR. The authors use a keyword lexicon to flag reviews that explicitly mention issues such as privacy, data sharing, consent, and related terms. They identify 118 SPR-related reviews inside the 15,090 negative-review corpus.

The authors are careful about interpretation: this method captures only explicit language. It likely misses implicit discomfort and may include occasional false positives. That caution is not a footnote-sized nuisance; it defines how the result should be used. The SPR analysis is a descriptive signal, not a full privacy-sentiment measurement system.

Still, the signal is strong. SPR-flagged reviews have a mean rating of 1.23, compared with 2.39 for non-flagged reviews. The reported comparison is statistically significant, with $t \approx 16.67$ and $p < .001$. These reviews also appear most often within billing and customer support issues, then access barriers and service unreliability, and least often within interaction quality.

The business reading is subtle. Explicit privacy complaints are not the most frequent negative-review theme. But when users do name privacy, security, or data handling, they are often already in a state of severe distrust. They question data collection at registration, logging, data sharing, storage, developer access, and whether the service’s privacy posture matches its behavior.

This suggests a conditional-trust model. Users may not scrutinize data practices while the product feels useful, fair, and reliable. Once they feel misled by billing, blocked by access, or disappointed by the service, they begin reevaluating the entire relationship. At that point, the privacy policy becomes less like assurance and more like evidence to be cross-examined.

For health AI operators, this is an unpleasant but useful insight. Privacy cannot be bolted onto a distrusted product by adding a longer policy. Data governance needs to be visible, comprehensible, and testable before trust collapses. That includes clear data-use explanations during onboarding, granular consent where practical, credible deletion and export options, and support staff capable of answering data questions without performing interpretive dance.

The paper does not prove that privacy concerns are caused by billing problems. It shows co-location: explicit SPR reviews cluster more in the billing/support category and carry lower ratings. Cognaptus’ inference is that privacy salience often rises when other parts of the product have already made the user suspicious. That is not a causal claim. It is a design warning.

The taxonomy turns into an operating dashboard

The cleanest business use of the paper is not to quote the abstract in a board deck. It is to convert the taxonomy into product instrumentation and governance review.

The three categories imply three different operating dashboards. Access and reliability should be tracked through blocked onboarding rate, login failure rate, crash frequency, payment-gate abandonment, support-seeking after onboarding, and feature-availability incidents. Interaction quality should be tracked through unresolved-intent rate, repeated generic-response detection, personalization failure, escalation appropriateness, user correction frequency, and post-interaction usefulness ratings. Billing and support should be tracked through trial-to-paid complaint rate, cancellation completion time, refund dispute rate, first-response time, resolution rate, and complaint recurrence by plan type.

Privacy and data governance need their own layer. Track data-practice questions, deletion requests, consent opt-outs, privacy-related support tickets, complaints mentioning logging or sharing, and negative-review clusters around data terms. More importantly, connect these signals to billing and access complaints. The paper suggests that privacy distrust may not appear as a standalone metric until another failure triggers it.

A practical version looks like this:

This is where AI governance becomes less mystical. A risk committee does not need to wait for a dramatic hallucination incident to discover that the product is unsafe as an experience. The system may already be producing distrust through mundane pathways: inaccessible functions, opaque pricing, useless replies, and evasive support. Nobody puts that on a glossy “AI safety” slide because it looks too much like operations. Naturally, that is where much of the work is.

What the evidence supports—and what it cannot carry

The paper is strongest as a large-scale, cross-app map of user-reported breakdowns. It captures complaints written in natural settings rather than in a controlled lab. It identifies recurring categories and gives enough numeric structure to compare category volume and rating severity. It also adds a useful privacy lens by showing that explicit SPR mentions, though uncommon, are associated with especially negative ratings.

But the evidence has clear boundaries.

First, the corpus is built from app-store reviews. People who write reviews are not a representative sample of all users. Negative reviews are especially likely to overrepresent strong dissatisfaction, recent frustration, or users motivated enough to complain publicly. That does not invalidate the study. It means the paper is about visible breakdown narratives, not total user experience distribution.

Second, the analysis is English-language only after filtering. The apps were collected without regional filters, but the retained text cannot fully represent non-English users or culturally different complaint patterns. This matters because health expectations, billing norms, regulatory protections, and privacy language vary by region.

Third, the negative-review filter relies on TextBlob sentiment classification. Sentiment tools can misclassify sarcasm, mixed reviews, domain-specific phrasing, or complaints wrapped in polite language. The review set is therefore a practical corpus, not a perfect sample of all negative experience.

Fourth, LDA topics and human groupings are interpretive. The authors chose a 10-topic solution based on coherence and interpretability, then grouped the topics into three categories. Another modeling setup might produce a slightly different taxonomy. The value here is that the taxonomy is coherent and operationally useful, not that it is metaphysically final.

Fifth, the privacy analysis captures explicit mentions only. Users may feel privacy discomfort without naming it as privacy, security, or data handling. Conversely, a keyword hit does not automatically mean a verified data-governance violation. The SPR result should be treated as a signal of expressed concern and distrust, not a forensic audit.

Finally, the paper does not measure clinical outcomes. It does not prove that specific chatbot failures caused harm, worsened symptoms, or changed healthcare decisions. It does show that users sometimes describe emotional frustration, worsened states, and rejection of services in relation to chatbot failures. That is enough to justify deeper safety and well-being research, but not enough to quantify harm.

These boundaries do not make the study weak. They make it usable. A business team should not use it to claim “18.4% of all chatbot users hate billing.” It should use it to ask why billing-related complaints, when visible, are so consistently severe.

The useful conclusion is boring in the expensive way

The paper’s conclusion is not that AI healthcare chatbots should vanish into a regulatory volcano. Nor is it that better models will solve everything. The more practical conclusion is that chatbot value depends on the alignment of access, interaction quality, financial trust, support, and data governance.

That sounds boring, which is usually how expensive truths enter the room.

For AI healthcare companies, the paper implies that model improvement is only one investment line. Product teams also need transparent onboarding, reliable access, honest capability claims, clear escalation paths, support processes that work after payment, billing flows that do not feel predatory, and data practices that users can understand before distrust forces them to investigate. None of this has the glamour of a new benchmark. It does, however, determine whether the benchmarked system survives contact with actual users.

For investors and enterprise buyers, the paper offers a diligence lens. Ask not only what model powers the chatbot, but how complaints distribute across access, interaction, billing, support, and privacy. Ask what the company measures when a user churns after onboarding. Ask whether privacy tickets rise after billing disputes. Ask how often paid features are described as generic. Ask whether the support team can explain the data lifecycle. The answers will be less theatrical than a demo and much more revealing.

For policymakers, the paper supports a broader view of consumer-facing digital health governance. If health chatbot apps sit between formal healthcare regulation and general consumer technology, then users encounter a gray zone where sensitive needs meet ordinary subscription mechanics. That gray zone is where opaque pricing, weak redress, and unclear data use become more than customer-experience defects.

The fashionable question is whether the AI can answer like a clinician. The paper asks something more operationally inconvenient: can the system around the AI behave like a trustworthy service? Until that answer is yes, the chatbot may fail before it answers.

Cognaptus: Automate the Present, Incubate the Future.