A credit model approves the familiar applicant. A diagnostic model reads the common scan. A pricing model values the house in a neighbourhood it has seen a thousand times before. Everyone relaxes. The model is “confident”.
Then a strange case arrives.
The applicant has an unusual income pattern. The scan comes from an underrepresented patient group. The house sits outside the areas covered by historic transactions. The model still produces an answer, because that is what models are trained to do. Press button, receive number. Very efficient. Occasionally ridiculous.
The harder question is not whether the model is uncertain. It is why it is uncertain.
Franc and Paplham’s paper, Epistemic Reject Option Prediction, tackles that distinction with unusual precision.1 It argues that reject-option systems — models allowed to say “I will not predict” — have been mixing together two very different reasons for refusal. One reason is that the world itself is noisy. The other is that the model simply has not seen enough relevant data. Those are not the same operational problem, and treating them as one bucket is a neat way to build a cautious system that is cautious about the wrong thing.
The paper’s contribution is a framework for epistemic rejection: abstaining not when the target is inherently unpredictable, but when available training data is insufficient to support a well-informed prediction. In business language, this is the difference between saying “this case is objectively volatile” and “we are outside our evidence base”. The first is a property of the environment. The second is a property of the model’s learning history. Confuse them, and your risk controls become theatre with equations.
Three ways an AI system can refuse to answer
The paper is best understood as a comparison among three reject-option logics.
| Rejection logic | What triggers refusal | What the refusal means | Business interpretation |
|---|---|---|---|
| Aleatoric rejection | High conditional risk from irreducible noise | “Even the best predictor would struggle here.” | The case is intrinsically noisy or volatile. |
| Bayesian total rejection | High total predictive uncertainty | “This prediction is uncertain overall.” | Useful, but diagnostically blunt: it mixes noise and ignorance. |
| Epistemic rejection | High expected conditional regret | “Our training data is not enough to justify this prediction.” | The model may need human review, more data, or a deployment boundary. |
Traditional reject-option prediction follows the first row. It rejects inputs where the expected prediction error is high. This makes sense when the data distribution is known or when the training set is large enough that epistemic uncertainty can be ignored. That assumption is comfortable in textbooks and ambitious in production.
Bayesian reject-option prediction improves the situation by using a posterior predictive distribution. It can represent both irreducible noise and uncertainty from finite data. But the refusal still comes from total uncertainty. Total uncertainty answers the question: “How uncertain is the prediction overall?” That is useful, but it does not say whether the problem is the world being noisy or the model being under-informed.
The proposed epistemic reject-option predictor asks a different question: “How much worse do we expect this learned predictor to be than the Bayes-optimal predictor that had full knowledge of the data-generating process?”
That expected performance gap is the paper’s central object. It is conditional regret.
In simplified form, the epistemic quantity is:
Here, $H^\ast(x, D)$ is the Bayesian predictor learned from finite training data $D$, while $h(x, \theta)$ is the Bayes-optimal predictor that would know the true parameter $\theta$. The model accepts the input if this regret is below a chosen rejection cost $\delta$:
That threshold has a clean interpretation. It is not a mystical confidence score. It is the maximum expected degradation one is willing to tolerate from acting with finite data rather than full knowledge. Management committees enjoy pretending they want interpretability. This is the kind of interpretability they might actually use.
The misconception: refusal is not always about noisy cases
The paper’s running intuition is simple. Suppose a house-price model has many examples from the city centre and few examples from distant suburbs. Prices in the city centre may vary widely because of unobserved factors: floor quality, renovation history, school catchments, view, sunlight, or the charming chaos of property markets. That is aleatoric uncertainty. Even with more data, some variation remains.
A suburb with little transaction history creates a different problem. The model may not know the local price function at all. That is epistemic uncertainty. More relevant data could reduce it.
An aleatoric reject-option predictor rejects the volatile city-centre cases. A Bayesian total-uncertainty predictor may reject both volatile city-centre cases and under-sampled suburbs. The epistemic reject-option predictor rejects the under-sampled suburbs while still allowing predictions in noisy but well-covered city-centre areas, because in those areas the model may already be about as good as any predictor can be.
That is the paper’s quiet but important correction. Rejecting noisy inputs is not the same as rejecting unsupported inputs. For business systems, the distinction matters because the remedy changes.
If the problem is aleatoric noise, the answer may be pricing buffers, probabilistic outputs, wider confidence intervals, or conservative decision thresholds. If the problem is epistemic uncertainty, the answer is different: collect more data, expand training coverage, route to human review, restrict deployment, or redesign the model class. One says “the world is messy”. The other says “our evidence is thin”. Only one can be fixed by buying more data.
The framework turns common uncertainty scores into decision rules
The paper’s theoretical move is not merely to rename epistemic uncertainty. It derives the rejection rule from a decision objective.
Standard Bayesian prediction minimises expected loss. The proposed epistemic reject-option framework instead minimises expected regret. That change sounds small, but it changes what abstention means.
The predictor itself remains the optimal Bayesian predictor. The difference sits in the selector — the part deciding whether to return a prediction or reject. Bayesian total rejection uses:
Epistemic rejection uses:
The paper then shows that familiar uncertainty quantities fall out as special cases. Under squared loss in regression, epistemic uncertainty becomes the posterior variance of the conditional mean. Under cross-entropy in classification, it corresponds to an expected KL divergence between parameter-conditioned predictive distributions and the posterior predictive distribution. Under 0/1 loss, it has another related form based on the expected maximum class probability.
This matters because practitioners already use entropy, variance, ensembles, dropout-based uncertainty estimates, and other Bayesian-flavoured heuristics. The paper does not merely say “these scores are useful”. It says: for this specific reject-option objective, these scores can be interpreted as conditional regret under particular losses. That is stronger than a dashboard metric with a nice colour gradient.
The main experiment validates the ranking objective, not production readiness
The main empirical test is a controlled synthetic polynomial regression task. That choice is deliberate, not a weakness to be apologised for every third paragraph.
To evaluate true regret, the authors need access to the data-generating process. Real-world datasets do not provide that. So the experiment samples ground-truth polynomial functions and generates labels with heteroscedastic noise. This lets the authors compute the relevant quantities analytically and compare rejection strategies fairly.
The experiment compares three strategies:
| Test | Likely purpose | What it supports | What it does not prove |
|---|---|---|---|
| Synthetic polynomial regression | Main evidence for the theorem in a controlled setting | Epistemic rejection gives the best regret-coverage trade-off across training sizes | That the method is ready for large deep learning systems |
| Risk-coverage comparison | Contrast with standard reject-option objectives | Bayesian total uncertainty is better when the objective is overall predictive risk | That total uncertainty is inferior in every practical use case |
| Training-size variation | Sensitivity to finite data | Epistemic uncertainty dominates in low-data regimes and shrinks as data grows | That all epistemic uncertainty disappears in misspecified models |
The key metric is the area under the Regret-Coverage curve. Coverage means the proportion of inputs accepted for prediction. Lower regret at a given coverage means the system is better at accepting cases where its finite-data predictor is close to the fully informed benchmark. The proposed epistemic strategy consistently achieves the lowest area under that curve in the synthetic regression experiment.
The paper also reports the Risk-Coverage curve, where the Bayesian total-uncertainty strategy performs best. This is not a contradiction. It is the point.
If the goal is to minimise total predictive loss on accepted samples, total uncertainty is the right ranking signal. If the goal is to identify where finite training data makes the learned predictor inadequate relative to the ideal predictor, epistemic regret is the right ranking signal. Different objectives, different abstentions. A system cannot be “best at uncertainty” in the abstract. It can only be best relative to the decision it is being asked to support.
That observation is more useful than another leaderboard. It tells deployment teams to stop asking whether a model is uncertain and start asking what kind of refusal they want.
The appendix is a realism check, not a second thesis
The appendix extends the validation to a more realistic image-based setting: age prediction from synthetic face images. The authors use StyleGAN3 to generate face images and define a ground-truth age distribution through a trained and distilled age-estimation model. This construction preserves access to the ground-truth mean and variance, which is necessary for calculating regret.
The learning model is Bayesian Linear Regression over fixed image features, implemented as a Gaussian Process with a linear kernel. The same three rejection strategies are compared: aleatoric, total, and epistemic.
There are three useful readings of the appendix.
First, in a well-specified setting using oracle aleatoric variance, the epistemic strategy again minimises regret, while the aleatoric strategy minimises risk. That reinforces the theoretical distinction: the epistemic predictor can accept noisy samples if it has learned the conditional mean well, while the aleatoric predictor rejects them because they remain noisy.
Second, when aleatoric variance must be learned rather than supplied by an oracle, the experiment becomes more brittle. Around the interpolation threshold, where the number of samples approaches the feature dimension of 2048, the paper observes a characteristic error peak. In the underdetermined regime, the estimated noise variance can collapse toward zero because the model can interpolate training data. Despite this instability, the epistemic strategy still maintains the lowest regret. This is a robustness-style test: it checks whether the signal survives a more realistic nuisance, not whether the entire production stack has been solved.
Third, the misspecification experiment is the most important boundary. When the true function lies outside the learner’s hypothesis class, epistemic uncertainty can vanish as data grows, while regret converges to a non-zero approximation error. In plain English: if the model class is wrong, more data can make the model confidently settle into the wrong shape. The proposed method targets estimation error from finite data, not approximation error from limited model capacity.
That limitation is not a footnote for lawyers. It is operationally central.
What this gives business teams: a better reason for escalation
The business value of epistemic rejection is not that an AI system becomes more humble. Humility is not an enterprise metric, though several vendor decks are bravely trying.
The value is diagnostic routing.
A high-stakes AI system often needs to decide among three actions: predict automatically, defer to a human, or request more information. Current confidence and uncertainty measures often blur the reasons for deferral. Epistemic rejection offers a cleaner escalation signal: this case is not merely difficult; it is under-supported by the training data.
That creates several practical uses.
In lending, epistemic rejection can flag applicant profiles that sit outside the model’s historic evidence base. This is not the same as saying the applicant is risky. It says the model does not have enough comparable cases to justify automated scoring. That distinction is important for fairness, compliance, and customer treatment.
In medicine, the framework maps naturally onto triage. A model may be allowed to predict for noisy but well-represented cases while deferring underrepresented patient profiles, rare presentations, or acquisition conditions outside its training coverage. Again, the refusal is not “this patient is hard”; it is “our model’s learned mapping is not adequately supported here”.
In insurance, epistemic rejection can separate volatile but familiar risks from unfamiliar risks. A familiar category with high variance may require pricing margin. An unfamiliar category may require underwriting review or new data collection. Mixing those responses is how automated decision systems become expensive filing cabinets with APIs.
In industrial quality assurance, epistemic rejection can mark new materials, new suppliers, new machine settings, or unusual sensor regimes as outside the evidence base. That is especially useful when the cost of a confident false pass is much higher than the cost of inspection.
The common pattern is simple:
| Model signal | Operational response |
|---|---|
| Low risk and low epistemic regret | Automate prediction |
| High aleatoric uncertainty but low epistemic regret | Predict probabilistically, price conservatively, or widen intervals |
| High epistemic regret | Escalate, collect data, restrict automation, or update training coverage |
| Persistent regret under low epistemic uncertainty | Suspect model misspecification, not merely data scarcity |
The last row is where many organisations will trip. If the system has enough data but remains wrong, the problem is not ignorance. It is model capacity, feature design, causal mismatch, distribution shift, or a business process pretending that regression is strategy.
The threshold is a policy choice hiding inside the maths
The rejection cost $\delta$ looks like a technical parameter. It is also a governance decision.
A low $\delta$ means the system rejects whenever the finite-data predictor is only slightly worse than the full-information benchmark. That gives conservative automation with more human review. A high $\delta$ allows more predictions but tolerates larger regret from limited evidence. The threshold therefore encodes the organisation’s tolerance for acting beyond its data support.
This is where the framework becomes practically interesting. Instead of debating whether a model is “confident enough”, teams can specify what kind of expected degradation they are willing to accept before deferral. That is not magically easy, but it is at least the right argument.
For regulated or safety-critical domains, the threshold could vary by decision type. A marketing recommendation can tolerate more epistemic regret than a loan denial, a diagnostic decision, or a machine shutdown recommendation. The paper does not provide a ready-made governance framework for setting $\delta$, and it should not pretend to. But it gives the parameter a meaning that risk teams can interrogate.
Where the paper stops
The paper is theory-first. That is a feature, but it defines the boundary.
The strongest results are in settings where the posterior and relevant uncertainty quantities can be computed cleanly. The main experiments are controlled because regret evaluation requires access to the true data-generating process. The image-based appendix is more realistic, but still synthetic in a crucial sense: the ground truth is constructed so that regret can be measured.
Practical deep learning deployment remains open. Bayesian neural networks usually require approximate inference: ensembles, dropout, variational methods, Laplace approximations, or other compromises. The paper connects its theory to widely used uncertainty measures, but it does not solve the engineering problem of reliable posterior estimation at scale.
The misspecification limitation is equally important. Epistemic rejection detects uncertainty due to finite data within the assumed model family. It does not guarantee detection of approximation error when the model family cannot represent the true function. A model can have low epistemic uncertainty because it has learned its own inadequate hypothesis class very thoroughly. Congratulations: the model is now wrong with tenure.
That means epistemic rejection should not replace model validation, subgroup testing, drift monitoring, stress testing, or causal scrutiny. It should sit beside them as a more precise abstention mechanism.
The strategic lesson: “I don’t know” needs a reason code
The paper’s deeper value is that it gives abstention a reason code.
Most organisations already understand that AI systems should sometimes defer. The harder part is knowing what the deferral means. A system that refuses because the data is noisy should trigger one response. A system that refuses because the training data is thin should trigger another. A system that stops refusing but remains wrong should trigger an even less pleasant conversation about model design.
Epistemic reject-option prediction formalises the second case. It tells us when the issue is not that the world is unpredictable, but that the model has not earned the right to speak.
That is a useful standard for AI governance. Not grand, not cinematic, not especially convenient. Just useful. The best automation systems will not be the ones that answer every question. They will be the ones that know which questions are outside their evidence base — and have the decency to stop typing.
Cognaptus: Automate the Present, Incubate the Future. :::
-
Vojtech Franc and Jakub Paplham, “Epistemic Reject Option Prediction,” arXiv:2511.04855, https://arxiv.org/pdf/2511.04855. ↩︎