Coverage Is Not a Range: RPS for Ordinal Conformal Prediction
TL;DR for operators A risk system for ordered outcomes should return a coherent low-to-high range, not merely a collection of plausible labels. Coverage matters, but it does not distinguish an adjacent miss from one several severity levels away. A narrow range can therefore look efficient while hiding a rare, costly outcome. The method examined here first attaches a calibrated set of plausible labels to each prediction, then evaluates ordered errors through cumulative probability across the label scale. Its ranked probability score produces candidate-label scores that fall toward a predictive median and rise afterward. Thresholding this V-shaped sequence yields nested, median-centered contiguous ranges, even when the classifier’s class probabilities are not unimodal. ...