When Conservatism Wins: Rare-Event Estimation Depends on What You Fear Missing

TL;DR for operators

A model-risk team estimating failures too rare for ordinary sampling has to decide which kind of error it can least afford. The benchmark shows that this choice can reverse which estimator looks best. :contentReference[oaicite:0]{index=0}

Under approximately symmetric penalties, GA-AMLS has much lower average SPB loss than QLD for the 1-layer and 4-layer models. When the loss instead makes underestimation much more costly, QLD sharply outperforms GA-AMLS—even though QLD systematically overestimates and GA-AMLS has lower bias.

The operational lesson is not to choose from a universal estimator leaderboard. First specify the cost of missed failures, false alarms, and the deepest-tail events; then choose the estimator and evaluation loss together. Parulekar and Liu’s Estimating Rare Events in Language Models with Proper Evaluation¹ shows why a conservative estimator can be preferable when missed rare failures carry the greater cost.

Rare failures are hard to measure before they are hard to manage

A model-risk team may care about an output that appears once in a million, once in a billion, or less often. At that scale, direct sampling quickly stops being a practical measurement strategy. A probability near $10^{-9}$ cannot be estimated reliably by running a modest test set and counting occurrences.

That creates a second problem. Once ordinary sampling fails, teams need an estimator, but “better estimator” is not a complete specification. A monitoring system may care primarily about numerical accuracy, or it may regard underestimating a dangerous event as far more costly than overestimating it.

The paper shows that these objectives can select different methods.

GA-AMLS searches for rare events in activation space

The target quantity is narrow and precise:

It is the probability that an input drawn from a specified distribution makes token $t$ the model’s deterministic next-token argmax. This is not a probability of an unsafe completion or an autoregressive trajectory.

GA-AMLS attacks the sampling problem by moving from discrete input space into the model’s continuous activation space. It expresses the extremely small target probability as a product of easier conditional probabilities:

Instead of trying to hit the final rare event directly, particles progress through adaptively chosen logit-margin thresholds. Activations are whitened, a diagonal Student-$t$ prior is fitted, and MALA is used to rejuvenate particles while respecting each level constraint.

The practical advantage is visible in the benchmark behavior. Input-space MHIS and ITGIS frequently return zero estimates in the deepest tail. GA-AMLS avoids that failure mode. It also avoids QLD’s independence-based recombination assumption, although it introduces its own dependencies on prior quality and MCMC mixing.

The paper characterizes GA-AMLS as lower-bias but higher-variance than QLD.

The leaderboard changes when underestimation becomes expensive

The most consequential empirical result is not that one estimator wins. It is that the winner changes with the evaluation objective.

Under Shifted-Power Bregman Loss with $\alpha=1.5$, which the paper treats as approximately symmetric, GA-AMLS has lower average loss than QLD on all three model depths, with especially large gaps on the 1- and 4-layer models:

Lower is better.

When $\alpha$ rises to 2, the loss places substantially more weight on underestimation. The ranking flips sharply:

This is easy to misread. QLD has not suddenly become the lower-bias estimator. The source reports systematic QLD overestimation, particularly in the deepest tail. Under a strongly underestimation-averse loss, that positive bias becomes useful: overestimation functions as a conservative hedge.

So the reversal is not evidence that accuracy itself changed. The scoring policy changed which type of error was expensive.

SPB Loss makes the cost model explicit

The paper’s second contribution is therefore as operationally relevant as the estimator.

Standard rare-probability losses create awkward tradeoffs. Some are undefined when an estimator returns zero. Some encode fixed underestimation asymmetry. Others lack proper-scoring-rule properties.

Shifted-Power Bregman Loss introduces three controls. A positive $\varepsilon$ keeps the loss finite at zero estimates. The parameter $\alpha$ adjusts the relative penalty for underestimation versus overestimation. The parameter $\gamma$ places additional weight on the smallest ground-truth probabilities.

That separation matters. A team running accuracy-oriented benchmarking can use approximately symmetric settings. A safety oversight process can raise underestimation aversion. A deep-tail audit can separately increase the weight on the rarest events.

The paper also reports a useful interaction: increasing $\gamma$ narrows QLD’s advantage under conservative evaluation because additional deep-tail weighting exposes the region where QLD’s positive bias is strongest.

What the benchmark supports operationally

For teams estimating rare model failures, Cognaptus draws three practical implications from the evidence.

First, define the cost of missed failures before choosing the estimator. If numerical fidelity is the primary objective, the benchmark favors GA-AMLS under approximately symmetric losses. If a missed rare event is substantially costlier than a false alarm, conservative positive bias may be desirable rather than defective.

Second, zero estimates deserve explicit scrutiny. An estimator returning zero for an event in the $10^{-9}$ to $10^{-5}$ range can create the appearance of absence where the true problem is insufficient sampling support. GA-AMLS provides one activation-space route around this structural failure.

Third, estimator benchmarking and risk policy cannot be cleanly separated. The loss function determines what counts as a good estimate. A ranking reported without its underestimation, overestimation, and tail-weighting assumptions is incomplete for operational use.

The evidence stops well before modern deployment conditions

The study provides controlled comparative evidence, not a deployment validation.

The models have only 1, 2, or 4 layers, with $d=512$. Inputs come from independent-token benchmark distributions rather than natural prompts. The event is a single deterministic argmax token rather than a stochastic multi-token completion.

GA-AMLS also carries technical uncertainties that matter outside the benchmark. Its fitted Student-$t$ prior can mismatch the true activation distribution. Finite MCMC budgets can leave incomplete mixing and particle dependence. Continuous activation samples may not correspond to realizable discrete input sequences. And the method is computationally heavier than QLD.

These boundaries prevent a stronger claim that activation-space splitting is already validated for rare failures in modern production LLMs.

Rare-event measurement starts with deciding which error matters

The paper changes the rare-event estimation question from “Which estimator is best?” to “Best under which error costs?”

That is a more demanding operational question, but it is also the one deployment teams actually face. GA-AMLS provides stronger low-bias estimation evidence under approximately symmetric accuracy objectives. QLD demonstrates why conservatism can dominate when the loss heavily prices missed failures.

The benchmark does not identify a universal winner. It shows why a universal winner is the wrong object to look for.

Cognaptus: Automate the Present, Incubate the Future.