Longer Yet Dumber: Why LLMs Fail at Catching Their Own Coding Mistakes

TL;DR for operators

Code review usually starts after code exists. FPBench argues that this is already too late.

The paper behind FPBench tests whether large language models can detect faulty premises in code-generation requests before obediently producing code from them.¹ The answer is awkward. Many models can identify the flaw when explicitly told to check the question first, but most do not do so proactively. They behave less like careful engineers and more like very fast interns with a tragic respect for bad tickets.

The operational message is simple: do not assume that a stronger coding model will automatically validate requirements. The paper’s strongest evidence is the gap between proactive error recognition and passive error recognition. DeepSeek-R1 leads proactive detection with a reported overall PRER of 0.57, while GPT-4.1 leads prompted detection with a PAER of 0.81. That gap matters because real users rarely announce: “Please inspect my assumptions before helping me implement this possibly defective specification.”

The second message is less comfortable for teams currently paying for longer reasoning traces: more tokens are not the same thing as better judgement. FPBench reports that faulty premises tend to inflate response length, especially when models are explicitly asked to scrutinise the prompt. Some models get wordier without becoming proportionally more accurate. The enterprise lesson is not “make the model think longer.” It is “force the workflow to validate premises before generation.”

FPBench is not a production software benchmark. It is built mainly from HumanEval and MBPP+ style tasks, then altered through controlled faulty-premise construction. That makes it useful for isolating a specific failure mode, but not sufficient for certifying behaviour in multi-file systems, legacy codebases, framework-heavy projects, or domain-specific engineering work. Still, the diagnosis is sharp: AI coding risk begins upstream of code. It begins with the model’s willingness to believe the user.

The model can find the flaw. It just usually waits to be asked.

The most important FPBench result is not that models fail. Everyone in software already has a small museum of AI-generated nonsense. The more interesting result is that failure is often conditional.

When the faulty prompt is presented normally, models frequently proceed as though the user’s premises are sound. When the same faulty prompt includes explicit guidance to inspect the premise first, detection improves sharply. This separates two capabilities that are too often blurred together:

This is why the PRER–PAER gap is the article’s spine. FPBench does not merely ask, “Can this model solve code tasks?” It asks, “Does this model know when the task itself is broken?”

That is a different bar. Passing unit tests is about behavioural compliance. Premise scrutiny is about epistemic hygiene. Less glamorous, admittedly. Also the part that prevents your assistant from elegantly implementing the wrong thing.

FPBench turns bad requirements into a measurable test

FPBench is built around “faulty premises”: prompt information that prevents a model from correctly understanding the intended coding task. The authors start from existing code-generation benchmarks, mainly HumanEval and MBPP+, then construct faulty versions of the tasks. The resulting benchmark contains 1,800 faulty-premise problems across three perturbation families.

The construction is important because not all bad prompts are bad in the same way.

The rule-based deletion design is especially revealing. The authors do not simply remove the most important premise. That would often collapse the task so obviously that even a mediocre model might complain. They also avoid deleting trivial information that might not matter. Instead, they target a premise that is important enough to create hidden logical damage while leaving the task superficially parsable.

That is a useful design choice. Real product requirements rarely arrive as a smoking crater. They arrive as plausible tickets with one missing edge case, one false assumption, one misleading comment, or one implied dependency nobody wrote down because apparently everyone enjoys archaeology.

The benchmark’s metrics separate obedience from scrutiny

FPBench uses four main metrics.

PRER, or Proactive Error Recognition Rate, measures how often the model independently identifies the faulty premise without any special instruction. PAER, or Passive Error Recognition Rate, measures how often the model identifies the fault after being explicitly told to inspect the premise. PROR and PAOR measure output-length overhead under faulty premises and under explicit guidance respectively.

The distinction matters because conventional code-generation metrics can be blind to the relevant failure. A model may produce syntactically plausible or even test-passing code while still accepting a premise that should have been challenged. In ordinary benchmarks, that can look like success. In real engineering, it is how teams get a beautifully formatted defect with comments.

FPBench’s metric design therefore shifts evaluation from “Did the model produce code?” to “Did the model deserve to produce code yet?”

That one word, yet, is the hinge. Mature coding assistants should not treat generation as the default first move. Sometimes the correct first output is a question. Sometimes it is a refusal to proceed until the requirement is repaired. Yes, that is less magical. So is having fewer production incidents.

The numbers show latent ability, not autonomous judgement

The overall results show a clear pattern: prompted detection is much stronger than unprompted detection.

The most business-relevant comparison is GPT-4.1: PRER 0.23, PAER 0.81. That is not a small prompt effect. It means the model can often detect faulty premises when placed in inspection mode, yet typically does not activate that mode by default.

O4-mini is even more revealing: PRER 0.12 and PAER 0.71. That is a large gap between “ability exists somewhere in the system” and “ability appears when the user needs it.” For an enterprise coding assistant, the gap is the product risk.

DeepSeek-R1 performs best on proactive recognition, with an overall PRER of 0.57, but even that leaves nearly half of faulty-premise cases undetected. This is not a solved problem hiding behind one underperforming model family. It is a behavioural pattern across model types.

The paper also reports type-specific differences. DeepSeek-R1 leads PRER across RUD, UPI, and RAD with values of 0.48, 0.59, and 0.64 respectively. GPT-4.1 performs much better under passive prompting, with PAER scores of 0.79, 0.86, and 0.79 across the same categories. This reinforces the central interpretation: models can be pushed into scrutiny, but many do not volunteer it.

Longer answers are not a substitute for better judgement

FPBench’s title-friendly finding is the overhead problem: faulty premises often make models longer, not necessarily wiser.

The paper reports that faulty and explicitly guided prompts tend to increase answer length. That is expected up to a point. If a model detects an inconsistent requirement, it may need to explain the issue, ask for clarification, or outline alternatives. Some extra text is not waste. It is the cost of not hallucinating with confidence.

The problem is that output length can become compensatory theatre. The paper highlights cases where passive scrutiny creates substantial inflation. GPT-4.1-mini reaches a PAOR of 2.42, O3-mini 2.35, and GPT-4.1 1.87. In plain language, some models generate much more text when told to inspect the prompt.

But the paper’s more useful point is not “long answers are bad.” It is that answer length must be tied to diagnostic yield. The authors argue that beyond a certain overhead level, additional verbosity yields diminishing gains and increases redundant output. This is the unpleasant little accounting fact behind reasoning-token optimism: there is a difference between inspecting a requirement and producing a ceremonial audit trail.

For operators, this suggests a practical design principle. Do not ask the model to “think step by step” forever and hope wisdom emerges from the invoice. Use structured checks:

1. Identify the required inputs.
2. Detect missing or contradictory premises.
3. Classify the issue as ambiguity, contradiction, missing dependency, or irrelevant perturbation.
4. Ask only the minimum clarification needed.
5. Generate code only after the premise passes validation or the uncertainty is explicitly accepted.

That is not as romantic as “autonomous agentic coding.” It is also more likely to survive contact with a sprint board.

Three faulty-premise types expose different failure modes

FPBench’s three perturbation families are not decorative benchmark variety. They are intended to activate different weaknesses.

RUD, the rule-based deletion method, tests whether the model can reason when an important premise is missing but the problem still looks structurally valid. This is close to many real requirements failures: the ticket is not nonsense, but it lacks a condition that changes the correct implementation.

UPI, unrelated perturbation insertion, tests whether the model can resist misleading text. This matters because code prompts often mix natural language, examples, comments, and hints. A model that overweights a plausible but false hint may implement the hint rather than the task. That is not reasoning. That is well-mannered gullibility.

RAD, random deletion, gives a less targeted missing-information stress test. It helps contrast deliberate high-impact deletion with more arbitrary prompt damage.

This is where the paper becomes more useful than a leaderboard. The models are not failing in one generic way. They are failing across several layers of input understanding: missing dependencies, contradictory language, and incomplete prompt structure.

That matters for product design because one guardrail will not cover all three. A “please be careful” system prompt is not a requirements-validation system. It is a Post-it note wearing a helmet.

What the paper directly shows, and what Cognaptus infers

The paper directly shows that FPBench can expose differences between proactive and prompted faulty-premise recognition across 15 LLMs. It directly shows that models often improve when explicitly instructed to inspect premises. It directly shows that response-length overhead rises in many scrutiny scenarios. It directly shows that different faulty-premise construction methods produce different performance patterns.

The business interpretation goes one step further.

The key inference is this: coding assistants need a pre-generation validation layer.

Not a bigger autocomplete box. Not a more encouraging system prompt. Not another dashboard showing pass@k on clean prompts. A real validation layer should treat the user request as evidence to be checked, not scripture to be executed.

How this changes enterprise coding-assistant design

For enterprises, FPBench is less a reason to abandon coding assistants and more a reason to stop deploying them as if the user prompt were clean input.

A production workflow should split the coding assistant into at least three stages.

First, the assistant should parse the request into premises: inputs, outputs, constraints, examples, dependencies, edge cases, and implied assumptions. This makes the hidden contract visible.

Second, it should run a premise audit. Are any variables undefined? Do examples contradict the requested behaviour? Does a comment imply one algorithm while the task requires another? Are units, return types, or boundary conditions inconsistent? This is the stage where FPBench’s lesson matters most.

Third, only after the audit should the model generate code, tests, or a patch. If the audit fails, the assistant should ask a targeted clarification rather than bluffing. A good coding assistant should occasionally be annoying in exactly the way a good senior engineer is annoying: early, specific, and cheaper than rollback.

This also changes procurement. Buyers should not ask only, “Which model writes the best code?” They should ask:

• How often does it challenge faulty requirements without being told?
• Can it detect contradictions between examples, comments, and stated behaviour?
• Does scrutiny increase cost predictably, or does it produce uncontrolled verbosity?
• Can its validation output be structured for CI, ticketing, or human review?
• Does it know when not to produce code?

The last question sounds philosophical. It is not. It is an operational control.

Where FPBench stops

FPBench is valuable because it isolates a neglected failure mode. That same isolation limits its coverage.

The benchmark is constructed mainly from HumanEval and MBPP+ style tasks. These are useful for controlled evaluation, but they do not represent the full mess of enterprise software. Real codebases involve multi-file context, stale documentation, build systems, framework conventions, permissions, integration contracts, migration history, and business rules with names like legacy_final_v3_REAL.

The faulty-premise types are also structured. RUD, RAD, and UPI cover important categories, but real requirements can fail through politics, incomplete product strategy, contradictory stakeholder goals, regulatory ambiguity, or hidden domain constraints. FPBench does not test whether a model understands the billing department’s interpretation of “active user.” A small mercy.

There is also a measurement boundary. PRER and PAER capture whether the model flags a faulty premise, but real deployment needs more: whether the flag is correct, whether the clarification is useful, whether the model avoids excessive false alarms, and whether developers actually follow the warning. A model that questions everything is not a careful engineer. It is a meeting.

Finally, overhead ratios based on answer length are informative but incomplete. Token count is a proxy for cost and verbosity, not a complete measure of cognitive efficiency. In production, the real overhead includes latency, reviewer attention, interrupted flow, and the cost of unresolved ambiguity.

The uncomfortable lesson: coding assistants need doubt engineered into them

FPBench’s contribution is not that it catches models making mistakes. That is now a mature genre. Its contribution is more specific: it catches models failing to question the premise that makes the mistake inevitable.

That distinction matters. A model that writes bad code from a good requirement is a code-generation problem. A model that writes plausible code from a bad requirement is a workflow problem. The second one is harder to notice because the output can look professional right up to the moment it does the wrong thing.

The paper’s most useful finding is therefore the PRER–PAER gap. It tells us that scrutiny is often present as a capability but absent as a default behaviour. In enterprise systems, capabilities that do not activate at the right time are not controls. They are decorations.

The next generation of coding assistants should be judged not only by how much code they can produce, but by how often they can say: this request is underspecified, this hint contradicts the goal, this variable is missing, this example does not match the stated behaviour, and no, I should not generate the implementation yet.

That is the move from completion to collaboration. It is also the move from “longer” to smarter.

Cognaptus: Automate the Present, Incubate the Future.