The Answer Looked Clinical. The Critical Steps Were Missing.

TL;DR for operators

A clinical assistant can follow instructions, produce a complete-looking answer, and read like expert work without performing the reasoning needed for a safe clinical decision. Those surface qualities may justify further evaluation, but they do not justify procurement or decision authority.

Across five deliberately difficult tasks, the models satisfied 80–90% of the least consequential criteria but only 32.4–41.7% of the most clinically consequential criteria. All three models missed 56 of 108 critical requirements. The systems were therefore strongest where omissions mattered least and weakest on reasoning and safety steps most closely tied to harm.

A single overall score can conceal that reversal of priority. Clinical evaluations should instead score each required inference, warning, action, or restraint separately, assign greater importance to items that can change patient management, and block release when a decisive requirement is missed.

The study supports this evaluation process more strongly than any model ranking. Its findings come from five difficulty-selected, single-turn tasks without retrieval, tools, retries, or reasoning scaffolds, so deployment teams should apply the method to their own clinical workflows and system architectures.

A convincing answer can still fail the clinical decision

Consider a team reviewing a medical assistant for procurement. The system extracts the relevant history, follows the requested format, discusses several plausible diagnoses, and produces an answer that reads like competent clinical work. In a demonstration, those qualities are reassuring because they are immediately visible.

The harder defect is an omission that does not disrupt the prose. The model may fail to reconcile evidence that contradicts an earlier diagnosis. It may identify a medication but not trace the interaction that makes it dangerous. It may recommend an intervention without recognizing that withholding it is the safer action. A reviewer reading for general quality can miss the same omission.

The operational stakes extend beyond benchmark design. They affect which products advance through procurement, which use cases receive clinical decision authority, which failures trigger release blocks, and where scarce expert-review time is allocated.

The pilot study by Samiha A. Ismail, Fan X. Chen, and Ali Merali was designed to expose precisely these omission failures.¹ Its most consequential result is not that one model achieved the highest mean score. It is that all three models were strongest on criteria with little clinical impact and much weaker on the criteria most closely connected to harm.

The evaluation scores decisions, not impressions

The authors constructed five synthetic scenarios covering anaesthesia, internal and family medicine, emergency medicine, and obstetrics. Practising clinicians wrote the cases and corresponding golden answers. Those answers were then decomposed into 184 separate requirements.

Each requirement was intended to capture one assessable inference or action. A criterion might require the model to identify a contradiction, connect a drug to a physiological mechanism, anticipate a missing resource, or avoid an unsafe intervention. The criteria were also assigned weights from 1 to 5 according to clinical impact.

This is the paper’s central methodological contribution: an atomic, clinically weighted rubric. “Atomic” means that each required step is scored independently rather than absorbed into a general impression of the answer. Weighting prevents a model from compensating for one critical omission by accumulating points for several low-consequence details.

The task score is the proportion of total available weight satisfied according to the final expert labels:

The design does not make expert judgment disappear. Clinician-written answers generated the initial criteria; rubrics underwent structural checks and independent clinical peer review; model graders screened the outputs; and experts reconciled cases where at least two graders disagreed. The final met-or-not-met decision remained an expert label.

That sequence matters for evaluation validity. Automated grading reduces the amount of material requiring close inspection, but it does not transfer clinical authority to the grader.

Performance declined as clinical importance increased

The clearest result appears when performance is separated by criterion weight.

The lower scores on weight-3 and weight-4 criteria are not perfectly monotonic, so the result should not be described as a smooth decline at every level. The broader pattern is still stark: criteria classified as trivial were usually satisfied, while fewer than 42% of critical criteria were met by any model.

This is a priority inversion. The systems were most reliable on details whose absence would matter least and far less reliable on the reasoning steps intended to distinguish a clinically adequate answer from a dangerous one.

Category-level results reinforce that interpretation. All three models received 100% for style and 87.5% for instruction following. Their reasoning pass rates ranged from 28.5% to 42.3%, while safety ranged from 38.8% to 44.8%.

Those numbers do not imply that style and instruction following are irrelevant. A clinical system must still communicate clearly and follow workflow constraints. They show that these visible qualities provide weak evidence about whether the underlying clinical reasoning is correct.

A product demonstration can therefore be accurate about what it shows and still be misleading about readiness. The system may genuinely be fluent, structured, and responsive. None of those observations establishes that it reliably detects the contraindication, updates its diagnosis, or escalates care in time.

Shared misses are more informative than the five-task leaderboard

The reported mean weighted scores were 0.47 for Claude, 0.39 for GPT, and 0.37 for Gemini. Claude led on three tasks, while GPT led on two.

That is insufficient evidence for a stable frontier-model ranking. The benchmark contains only five tasks, and inclusion was deliberately restricted to scenarios where at least two models scored below 0.60. The sample was built to examine difficult cases, not to estimate average clinical performance. The authors accordingly treat the differences as descriptive rather than statistically inferential.

The shared failures provide a stronger operational signal. Across the benchmark, 56 of 108 weight-5 criteria—52%—were missed by every model. These misses clustered around several recurring demands:

• reconciling later evidence with a salient earlier diagnosis;
• tracing multi-step disease and medication mechanisms;
• anticipating resource and transfer constraints;
• balancing competing risks under time pressure;
• recognizing when not intervening is the safer decision.

The obstetric scenario illustrates the magnitude. It combined pre-eclampsia, severe asthma, twin pregnancy, and the relative contraindication of labetalol. Weighted task scores ranged from 0.07 to 0.12, and 33 of its 38 critical criteria were missed by all three models.

The paper treats these task results as main evidence about failure patterns, not as an ablation or robustness exercise. The controlled harness held the prompts, rubrics, weights, scoring, and interaction format constant across models. This makes the common omissions easier to attribute to the tested systems rather than to different evaluation conditions, while leaving open how other deployment architectures would perform.

For a clinical AI team, a universally missed critical criterion has immediate value. It can become a regression test after a model update, a red-team scenario for a new workflow, a release-blocking requirement, or a training example designed around the omitted inference.

Autoraters can narrow expert review, not replace it

Criterion-by-criterion clinical review is expensive, especially when every model response must be checked against dozens of requirements. The paper tests whether model graders can help route that workload.

Raw agreement with final expert labels reached 92.8% for the GPT autorater, 94.0% for the Gemini autorater, and 94.7% for the Claude autorater across 552 criterion-model evaluations.

These figures support automated disagreement screening. They do not establish autonomous clinical grading. Raw percentage agreement can look high when one label is common, and the paper does not report prevalence-sensitive measures such as kappa or macro-F1. Nor did every criterion receive fully independent grading from multiple clinicians.

A defensible workflow is therefore asymmetric:

1. Model graders apply the rubric at scale.
2. Agreement patterns determine which evaluations require attention.
3. Experts reconcile disputed or clinically sensitive labels.
4. Expert decisions remain the source of record.

The automation reduces search cost. It does not acquire clinical authority.

Turn critical criteria into deployment controls

The paper directly demonstrates a benchmark-construction and quality-control process. Cognaptus infers several uses for clinical AI procurement and deployment.

The affected user is the team deciding whether a clinical assistant may influence triage, medication, transfer, or escalation decisions. The relevant condition is a workflow where omission of one inference can materially change patient management. Under that condition, the release gate should operate at the criterion level rather than the response level.

This also changes how vendors should be compared. A buyer does not need a larger collection of polished sample answers. It needs evidence that the deployed system satisfies locally defined, high-impact requirements under the same retrieval, tooling, escalation, and human-review conditions it will encounter in production.

The results describe a hard single-turn baseline

Three boundaries constrain application of the findings.

First, five difficulty-selected tasks cannot estimate average clinical capability or support reliable general rankings. A larger benchmark could change both the mean scores and the apparent ordering.

Second, the rubrics rely mainly on UK and US clinical guidance. Organizations operating under different protocols, formularies, referral pathways, or resource constraints would need to localize both the scenarios and their weights.

Third, the evaluated systems received one API turn with no retrieval, tools, retries, or agentic scaffolding. This conservative setup isolates model responses under a common harness, but it does not represent every clinical product. Retrieval may improve access to guidelines. Tools may support calculations and interaction checks. Structured retries may recover omitted steps. They may also introduce new failure modes.

Deployment teams should therefore adopt the evaluation method without assuming that the reported scores transfer unchanged. The benchmark criteria should be rerun against the actual system architecture.

Evaluate what can change the patient’s path

The most reusable contribution of this pilot is a shift in the unit of evaluation. The relevant object is not the answer’s general quality. It is each inference, warning, action, or restraint that can alter the clinical decision.

Once those elements are separated and weighted, surface competence loses its ability to conceal a decisive omission. Model comparisons become more diagnostic, expert review becomes more targeted, and release decisions can be tied to explicit clinical consequences.

The paper does not identify a generally safest frontier model. It provides a disciplined way to find where several capable models fail together—and to prevent a well-written answer from receiving more authority than its clinical reasoning has earned.

Cognaptus: Automate the Present, Incubate the Future.