The Verdict Is Not the Evidence
TL;DR for operators In Beyond QASecClaw: Agent Scaling, Verifier Feedback, and Evidence Reliability in Software Assurance1, additional LLM assessment contexts did not consistently make defect-clearance decisions safer. Scaling from one to 50 contexts increased token consumption approximately 49-fold on eight authored Java warning controls without meaningfully reducing unsafe clearance. Raw formal-verifier feedback improved correct assertion verdicts but also increased incorrect ones. More surprisingly, models frequently rejected valid counterexamples that correctly demonstrated program defects. ...