An 8/10 Is Not a Probability: Validating LLM Confidence Before It Controls Workflow

TL;DR for operators

A model saying “8/10 confident” does not mean its underlying uncertainty is approximately 20%. Across the evaluated settings, the average instance-level correlation between reported confidence and logits-based confidence is only 0.135.

The more useful operating rule is narrower. First test whether reported scores vary enough to distinguish cases. Then measure whether those scores rank examples meaningfully on held-out data. Separately test whether their numerical scale agrees with the comparison signal and whether they are calibrated against correctness. Do not substitute one test for another.

For black-box deployments, validated confidence may still help with ranking, weighting, or routing. For white-box systems, disagreement between high reported confidence and lower internal confidence can itself become a warning signal. Neither use requires pretending that the reported number is a probability.

A precise-looking score can control a workflow without measuring what you think

Consider a production system that asks a model for an answer and a confidence score. An 8/10 answer is accepted automatically; a 5/10 answer goes to another model; a 3/10 answer reaches a human reviewer.

That architecture is easy to implement. Its reliability depends on a harder question: does the reported score actually track a distinct internal confidence signal?

Hefan Zhang and colleagues study that question directly in When Linguistic and Internal Confidence Diverge in Large Language Models.¹ Across 30 LLaMA-, Mistral/Mixtral-, and Qwen-family models and ten tasks, they compare user-facing confidence with logits-based confidence for classification and with semantic-entropy-based uncertainty for generation.

The headline number is modest: average instance-level Pearson correlation between reported and logits-based confidence is 0.135. A much stronger correlation appears when averages are pooled across model-task settings, but that aggregate relationship does not make individual scores reliable enough for case-level decisions.

That distinction matters when confidence decides which individual answer gets accepted, rerouted, escalated, or weighted.

Confidence reliability has three separate failure modes

The paper’s most consequential measurement choice is to separate three questions that are often compressed into one.

These properties do not move together.

Instruction-tuned models illustrate the problem. In paired comparisons, their average Pearson correlation is somewhat higher than that of base models—0.172 versus 0.118—but the difference narrowly misses the conventional 0.05 significance threshold ($p=0.054$). At the same time, instruction-tuned models show significantly larger cross-channel distance, 0.271 versus 0.241, and substantially worse linguistic-confidence ECE, 0.287 versus 0.191.

So a model can become somewhat better at ranking cases by confidence while becoming worse at expressing confidence on an interpretable numerical scale.

For an operator, that rules out a common shortcut. Testing correlation alone cannot justify treating “0.8 confidence” as an 80% probability.

The shape of the confidence scores explains more than the model label

The paper then asks why alignment differs so much across settings. Its regression analysis is descriptive rather than causal, but the recurring predictors are revealing.

Across 363 model-task-prompt settings, the reported OLS models explain 49.8% of variation in correlation, 57.0% in cross-channel distance, and 64.4% in confidence-to-confidence ECE. Distributional properties of the confidence scores repeatedly matter; model family, version, and type are generally less consistent once other variables are controlled.

One statistic is especially instructive: the standard deviation of reported confidence.

Greater dispersion is positively associated with correlation, with standardized $\beta=0.194$. But that same dispersion is also associated with greater distance, $\beta=0.780$, and higher cross-channel ECE, $\beta=0.318$.

This is the basis for the paper’s lossy-channel interpretation. When a model maps many different internal confidence states to 9s and 10s, distinctions disappear. Broader score dispersion can preserve more ordering information. Yet spreading scores across the scale does not ensure that the scale is correctly positioned, linear, or calibrated against correctness.

The result is a useful replacement for “more confidence variation is better”: variation can restore ranking information without restoring probability meaning.

Conversational framing can make confidence stronger and less grounded

The prompt experiments function primarily as robustness tests of the reported-confidence channel.

Criticism, approval, and even irrelevant contextual cues all increased average reported confidence in the tested conditions. At the same time, each significantly reduced its correlation with logits-based confidence. For example, approval raised mean confidence from 8.428 to 8.807 while reducing correlation from 0.266 to 0.130.

An explicit instruction telling the model to align its confidence with final-layer probabilities did not significantly improve the relationship.

This matters for systems in which confidence is elicited inside a conversation rather than through a fixed evaluation prompt. Confidence can move because the surrounding language changes, even when the movement does not represent improved internal alignment.

The confidence-elicitation ablation provides a separate implementation check. Removing the request for confidence did not change normalized task accuracy. Some logits statistics shifted for base models, while corresponding changes were not significant for instruction-tuned models. The result does not prove that elicitation is behaviorally inert, but it weakens the simpler explanation that the main accuracy results arise because asking for confidence fundamentally changes task performance.

Use confidence for the job it has actually passed

The paper does not conclude that reported confidence is useless. Its evidence supports a more conditional policy.

For black-box models, Cognaptus would treat the confidence score as a feature requiring task-matched validation. Before it controls a workflow, test at least three things:

1. Dispersion: Does the model produce enough variation to distinguish cases, or are scores collapsed near the top?
2. Held-out association: Do higher scores reliably rank cases by correctness or another relevant confidence proxy?
3. Calibration: If the application interprets the score probabilistically, does empirical correctness support that interpretation?

These tests correspond to different operational permissions. A score with useful association but poor calibration may support reranking or weighting. It should not automatically set a probability threshold for escalation.

For white-box systems, there is an additional option. High reported confidence combined with relatively low internal confidence can be treated as disagreement between two signals. That discrepancy could trigger additional sampling, reranking, abstention, or review without assuming either channel is ground truth.

The paper’s small gpt-4.1-nano aggregation experiment illustrates the ranking case. Majority voting reaches 0.7311 accuracy, level-confidence weighting 0.7531, and numeric-score weighting 0.7830. This is an exploratory downstream extension, not the main evidence base: it uses one model outside the main suite and cannot establish general confidence calibration. What it shows is narrower—validated rank-order information can sometimes improve weighting.

The evidence supports a validation policy, not a universal confidence model

Several boundaries constrain deployment claims.

First, logits-based confidence and semantic entropy are proxies for internal confidence, not direct measurements of a model’s true uncertainty. The generation analysis also depends on sampling and NLI-based semantic clustering choices.

Second, the regression results do not identify a causal training mechanism. Instruction tuning, model size, dispersion, and saturation are associated with different confidence patterns, but the paper cannot establish which training changes produced them.

Third, the main suite covers three open-model families. The subsampling analysis strengthens confidence that the 200-example evaluations reproduce larger-pool trends—the median absolute differences are roughly 0.024–0.027, with more than 93% of estimates within $\pm 0.10$—but it does not expand the model or task universe.

Those boundaries still leave a clear governance rule: do not grant a self-reported confidence score operational authority merely because it is numeric.

A model can sound highly certain, preserve some ordering information, disagree with its internal signal, and remain poorly calibrated at the same time. Production systems should decide which of those properties the workflow actually needs, measure that property directly, and constrain confidence-driven automation to the uses that validation supports.

Cognaptus: Automate the Present, Incubate the Future.