The Verdict Is Not the Evidence

TL;DR for operators

In Beyond QASecClaw: Agent Scaling, Verifier Feedback, and Evidence Reliability in Software Assurance¹, additional LLM assessment contexts did not consistently make defect-clearance decisions safer. Scaling from one to 50 contexts increased token consumption approximately 49-fold on eight authored Java warning controls without meaningfully reducing unsafe clearance. Raw formal-verifier feedback improved correct assertion verdicts but also increased incorrect ones. More surprisingly, models frequently rejected valid counterexamples that correctly demonstrated program defects.

For security engineering teams, the central distinction is between a model’s judgment and independently validated evidence. Evaluate unsafe clearance, benign clearance, unresolved decisions, evidence applicability, benchmark coverage, and inference cost together. Retain deterministic admission checks where reliable scope rules and validated evidence already exist. The study supports this design principle in its tested settings, not a general verdict against interactive multi-agent systems.

More assessments did not make the clearance decision safer

A security scanner flags a possible defect. The engineering team must decide whether to dismiss the warning, retain it for investigation, or leave the assessment unresolved. Dismissing a genuine defect creates risk; retaining every benign warning consumes engineering capacity.

Adding several AI specialists seems like a reasonable way to improve that decision. Each examines the same code from a different prompted perspective, and their collective judgment determines the disposition.

The experiment challenges the expected return from that arrangement.

Using Qwen2.5-Coder-7B-Instruct, the researchers increased assessment contexts from one to 50 on eight authored Java null-pointer warning controls. Total token consumption rose from 25,938 to 1,277,612, approximately 49 times as much, without a meaningful reduction in unsafe clearance (Section IV-A, Figure 2).

These contexts were fresh, single-turn assessments sharing the same base model and task evidence. They did not communicate, operate tools interactively, or repair code. A strict majority determined whether a warning was retained or cleared; other outcomes remained unresolved.

That distinction matters. The experiment measures repeated, role-prompted judgment rather than a broader architecture in which heterogeneous agents independently gather evidence.

The initial development comparison illustrates the failure. Across three seeds and 12 assessments of genuine defects, both one-context and five-context configurations incorrectly cleared 11. Both correctly cleared all 12 benign assessments. A verifier-only baseline already classified the eight controls correctly in every seed (Section IV-A, Table III).

The 1.5B model’s scaling sensitivity test was no more encouraging: it cleared every defect in every reported configuration. An adaptive-routing variant also activated all five roles for every development warning, leaving the benefit of genuinely task-dependent routing untested.

These results do not establish that increasing agent counts is generally ineffective. They establish that adding more homogeneous assessments, under this fixed-evidence procedure, produced little measurable assurance benefit despite substantial additional inference expenditure.

The benchmark exposes three competing outcomes

Assessing a warning-triage system requires distinguishing three outcomes:

• Unsafe clearance: a genuine defect is incorrectly dismissed.
• Benign clearance: a genuinely benign warning is correctly dismissed.
• Unresolved assessment: no usable disposition is reached.

Reducing unsafe clearance alone is insufficient. A system that retains everything avoids incorrectly dismissing defects but provides no warning-reduction benefit. A system that abstains frequently can also appear conservative while returning little actionable information.

The researchers evaluated these trade-offs on a stratified sample of 160 OWASP Benchmark cases: 100 real vulnerabilities and 60 benign cases.

Values separated by slashes represent seeds 17 and 43. The ten-context configuration assessed only 96 of 160 cases per seed; its unresolved counts include 64 unassessed cases per seed. Source: Section IV-A, Table IV.

Five contexts cleared more genuine vulnerabilities than one context in both seeds. Benign clearance varied, while unresolved assessments increased. There was no consistent improvement across the three outcomes.

The ten-context figures are particularly easy to misinterpret. Their lower unsafe-clearance counts do not establish safer decisions on equivalent coverage: 40% of the selected cases were never assessed in that arm.

Sampling also constrains extrapolation. The selected OWASP set deliberately included 40 real vulnerabilities without matching scanner reports. A clear-without-signal policy consequently had 40% unsafe clearance on the selected real cases, compared with 2.8% across the complete 2,740-case benchmark. These are different evaluation populations, not competing estimates of the same deployment rate.

For a security-tool buyer, incomplete assessments should therefore be reported separately from actual model abstentions. Otherwise, missing decisions can be mistaken for improved caution.

Verifier feedback improved answers and increased errors

Formal verification offers a different route to assurance: a checker can analyze a property and potentially produce evidence that the property holds or a counterexample demonstrating its violation.

The researchers asked whether supplying raw JBMC bounded-model-checker feedback improved model judgments on selected SV-COMP Java assertion tasks.

Across 32 frozen tasks, the mean number of correct model verdicts increased from 12.7 to 22.0 per seed when raw feedback was supplied. But incorrect verdicts also increased, from one to seven (Section IV-B, Figure 3).

Feedback helped the model reach more answers. It did not ensure those answers were reliable.

A deterministic JBMC-plus-JVM validation policy returned 26 correct and six unresolved verdicts on the same 32-task subset. In a broader backend calibration over 111 eligible tasks, 64 properties were established, 25 counterexamples were replayed successfully on the JVM, and 22 outcomes remained unresolved. All 89 decisive backend results agreed with expected verdicts.

The comparison has an important asymmetry: the deterministic policy performed JVM replay and property-specific validation that were not included in the model’s raw feedback. It is therefore not a controlled demonstration that deterministic reasoning outperforms model reasoning given identical validated evidence.

What it shows is narrower and operationally relevant. Giving an LLM solver output is not equivalent to giving it a validated result with checked scope, witness behavior, and verification obligations.

A valid counterexample can be evidence of an unsafe program

The most distinctive failure appears when models must decide whether formal evidence is applicable.

Consider a checker that produces a concrete execution causing a null dereference. If that execution corresponds to the intended program revision, property, and assumptions, it can be valid evidence of a defect.

The program is unsafe, but the evidence is applicable.

Those are separate judgments.

In the study’s evidence-applicability experiment, models rejected 70–100% of clean, applicable counterexample packages across evaluated configurations (Section IV-C, Figure 4). Some inspected responses treated the existence of a real defect as a reason to reject the evidence demonstrating that defect.

That is not merely excessive caution. It confuses the question “Does this evidence apply?” with “Is this program safe?”

The researchers also tested corrupted evidence packages involving scope mismatches, missing obligations, and other construction-specific alterations. Without deterministic gating, models accepted 0–3% of corrupted packages; observed erroneous acceptances involved suppressing harnesses or omitted obligations.

Raw verifier feedback did not resolve the clean-evidence rejection problem. Rejection reached 87% on development templates and 100% on held-out templates in the reported feedback condition.

The evaluation does not prove that applicability confusion explains every rejection; the mechanism was identified in inspected responses rather than independently measured across all errors. Clean packages were also all counterexamples, while corrupted packages reported solver success, so applicability and verifier-outcome type were not fully separated experimentally.

Nevertheless, the measurement point is significant: an assurance system must be evaluated for rejecting valid defect evidence, not only for accepting invalid evidence. A model can reject almost everything and still perform poorly at evidence adjudication.

Deterministic gates can block unsupported decisions, not recover missing ones

A deterministic evidence-admission gate checks specified conditions before permitting a proposed decision. Instead of trusting the model’s interpretation alone, it can require consistency in program revision, property identity, input-domain coverage, harness construction, and verification obligations.

The study reused identical model outputs for gate-on and gate-off comparisons, isolating the effects of postprocessing without generating additional model judgments.

Across repeated E1/E4 assessments, the gates removed 161 unsafe-clearance decisions. In E5, they removed 12 erroneous evidence acceptances across configurations. These are repeated decisions, not 173 independently discovered defects or new verification results.

For E5’s scope-validation tests, the construction-specific validator accepted all 20 clean packages and rejected all 100 corrupted packages in each of two template sets. The second set used disjoint program templates, making it a limited replication rather than simply another scoring pass.

Yet this success has two boundaries.

First, the validator used known metadata, canonical harness text, and established assumptions. It was not demonstrated to recognize arbitrary semantically equivalent harnesses or independently authored formal evidence.

Second, a gate that vetoes unsupported acceptances does not repair a model’s rejection of valid evidence. It can stop a dangerous clearance without generating the useful, correct decision the model failed to provide.

The architecture lesson is consequently conditional: deterministic checks can control evidence admission when reliable rules and underlying evidence already exist. They do not substitute for discovering defects, constructing proofs, or improving the model’s interpretation of legitimate counterexamples.

What security teams should require before deployment

Cognaptus infers an evaluation and architecture priority from these results.

For engineering leaders deciding whether to purchase or expand an AI-assisted warning-triage system, the acceptance test should jointly measure incorrect dismissal of real defects, correct dismissal of benign findings, unresolved decisions, unassessed cases, and total inference cost. Increasing assessment-team size should be evaluated against smaller configurations under equal total budgets and equal case coverage.

For teams incorporating formal verification, raw solver feedback should be distinguished from replayed counterexamples and property-specific validated results. Where scope and obligation checks can be implemented deterministically, final clearance or evidence acceptance should depend on those checks rather than on model agreement alone.

For developers building evidence-review assistants, evaluations should include both corrupted evidence that must be rejected and applicable defect evidence that must be accepted. This tests the distinction between evidence validity and program safety directly.

A separate warning-free test-generation experiment provides a related caution. No generated specification detected the target defects in three Defects4J Cli bugs through the required buggy-fails/fixed-passes differential. Strict parsing rejected all single-context outputs and 41 of 45 five-context outputs; a secondary parser recovered executable specifications without recovering discovery. One target operation was outside the adapter’s supported API (Section IV-D, Table V). The result is a negative finding for this restricted interface, not a general impossibility result for AI-assisted bug discovery.

Where the findings stop

The study records 13,638 completed model requests, but these are not independent defect observations. Assessments share seeds, programs, templates, and evidence packages.

Its architecture comparisons lack equal inference budgets; the OWASP ten-context arm has incomplete coverage; only Qwen2.5-Coder models were tested; and the principal development warnings originate from one authored semantic template. The evidence validator depends on known constructions. No significance tests, independent implementation audit, or public reproduction package establish broader reliability claims. A prepared family-disjoint Juliet split has no reported held-out model evaluation.

These limits prevent conclusions about interactive, heterogeneous, tool-using multi-agent systems or independently generated formal proofs.

The relevant next evaluation is not simply a larger context count. It is a budget-matched, equal-coverage comparison in which models receive appropriately validated evidence and must separately judge its applicability and the program’s safety.

The present results show why that distinction matters. Additional model verdicts can increase expense and confidence without improving the underlying clearance decision. Assurance depends on whether the evidence actually justifies what the system is permitted to conclude.

Reference

Cognaptus: Automate the Present, Incubate the Future.