Better Bounds, Later Arrivals: The Route-Choice Calibration Problem

TL;DR for operators

A delivery system can produce more reliable travel-time estimates and still choose routes that miss deadlines more often. In a 2026 traffic-routing study by Jianing Long and coauthors,¹ calibration improved simultaneous coverage of three candidate routes in Beijing and Chengdu, yet increased realized lateness under a rule that selects the route with the smallest upper travel-time estimate.

The practical distinction is between how well predictions describe uncertainty and how well decisions use those predictions. For navigation and logistics teams, an upgrade that improves coverage is not necessarily an upgrade in service reliability. Acceptance tests should measure deadline misses, travel time, and changes in route selection alongside prediction quality. The study establishes this distinction in offline proxy tasks, not live transportation deployments.

Better statistical coverage, worse routing outcomes

A delivery service must choose among several routes before knowing the traffic conditions it will encounter. Each route has an estimated travel time and an upper estimate intended to account for uncertainty. Choosing the route with the smallest upper estimate can appear sensible when the operator wants to avoid unexpectedly long journeys.

Suppose the forecasting system is updated so that its upper estimates cover actual travel times more reliably. Should the delivery service expect fewer late arrivals?

The paper examines this question using three fixed candidate paths for each origin–destination pair. It distinguishes joint candidate-set coverage—the proportion of cases in which all three realized route times simultaneously fall below their respective upper estimates—from the lateness of whichever route the system actually selects.

Those are different quantities. A system can become better at enclosing all possible route times without becoming better at selecting the route that meets a particular deadline.

The three formal comparisons illustrate the distinction.

Source: Section 6.2, Table 2. C0 uses raw 90th-percentile upper bounds; C2 uses jointly calibrated upper scores. Positive changes in lateness and travel time are adverse.

In Beijing M1, the calibration procedure moved joint coverage closer to the nominal 90% target. Chengdu M2 also approached that target, while Chengdu M1 improved substantially but remained below it. All three comparisons nevertheless recorded worse route-choice outcomes under C2.

The lateness changes correspond to approximately 16, 78, and 92 additional late outcomes per 10,000 evaluated proxy decisions, respectively. These are percentage-point changes, not relative percentage increases or estimates of additional late deliveries across either city.

The authors report paired 95% moving-block bootstrap intervals that exclude zero for each lateness increase. That supports the direction of the differences within the evaluated tasks, although the intervals do not establish how the results would generalize beyond the observed traffic dates.

The calibrated bound changes which route wins

To understand the result, the evaluation must be followed from forecast construction to route selection.

The study uses processed road-speed datasets from Beijing and Chengdu. Each city has 150 fixed origin–destination pairs, three candidate paths per pair, and 14 test dates. Forecasts and correlated residual scenarios generate distributions of possible travel times, calculated by summing distance divided by simulated speed across the links in each path.

Two upper-bound constructions are compared.

C0 takes the 90th percentile of each route’s simulated travel-time distribution. C2 adjusts that percentile using a coefficient estimated from calibration-period data and a path-specific uncertainty scale:

Here, $q_{0.9}(T_p)$ is the raw 90th-percentile path travel time, $\alpha$ is the fitted calibration coefficient, and $s_p$ measures dispersion in that path’s scenario distribution. The coefficient is fixed before testing.

The route-selection rule remains unchanged: choose the candidate with the smallest upper score.

The critical feature is that the adjustment differs across paths. If one route receives a larger uncertainty adjustment than another, their relative rankings can reverse. Calibration can therefore change the selected route even though the available candidate paths and replayed traffic outcomes remain fixed.

This is not merely an issue of all estimates becoming more conservative. In an ablation reported in Appendix A, replacing the path-specific scale with a constant produced no route reordering and no decision effect. The diagnostic supports differential score expansion as the relevant mechanism.

The main comparisons also show substantial expansion in upper scores relative to scenario medians. In Chengdu M1, the mean upper-score distance increased from 3.999 to 8.971 minutes. Larger bounds helped achieve greater coverage, but coverage alone did not establish whether the revised path rankings were preferable.

Route switches account for the additional lateness

The ranking mechanism becomes visible when the authors examine decisions that changed between C0 and C2.

Route-change rates were 2.89% in Beijing M1, 19.10% in Chengdu M1, and 16.62% in Chengdu M2. Most decisions therefore remained unchanged, but the switched decisions were sufficient to alter aggregate lateness.

The paper classifies a switch as harmful when a route arriving within budget under C0 is replaced by a route that exceeds the budget under C2. A beneficial switch produces the opposite change.

Source: Section 6.4, Table 4. These are matched offline decision counts, not independent observations of driver behavior.

In each comparison, harmful switches exceeded beneficial ones. Their difference accounts arithmetically for the increase in aggregate lateness.

Route changes were also concentrated near decision boundaries. In Chengdu M1, 57.9% of switches occurred in the lowest quartile of gaps between the two smallest raw upper scores; Chengdu M2 recorded 60.3%. A separate, post-result Beijing E168-CDF diagnostic reported 83.2%, but that supplemental branch is not interchangeable with the formal Beijing comparison.

These findings identify where operators might investigate unstable decisions: cases where competing routes have nearly equal upper scores and modest calibration adjustments can change the winner.

They do not establish that the observed switches would produce the same effects in live traffic. The decomposition is an accounting explanation within fixed replayed outcomes, not an identified behavioral or network-level causal mechanism.

A different routing objective changes the answer

The calibration results concern a specific decision rule: minimize the estimated upper travel time. That rule is not mathematically equivalent to minimizing average journey time or the probability of missing a delivery deadline.

The study evaluates alternative interfaces using the same frozen scenarios and budgets. In the formal Chengdu M2 comparison, selecting the route with the lowest scenario-mean travel time produced a 16.8095% lateness rate, compared with 17.4705% for C0 and 18.3905% for C2.

A hypothetical oracle using realized future travel times achieved 12.8305%, indicating additional improvement was possible within the three-path candidate set. The oracle is not deployable because the route selector cannot know actual future travel times when making its decision.

The scenario-mean result does not establish that mean minimization is universally superior. It demonstrates that the choice of optimization objective materially affects measured performance.

The study also reports a separate Chengdu forecasting comparison in which M2 reduced speed mean absolute error by 14.91% relative to M0. Under the unchanged C0 selection interface, lateness was 1.4571 percentage points lower and mean travel time fell by 0.1250 minutes. Better point accuracy can therefore accompany better routing outcomes, even though improved uncertainty coverage did not do so in the principal calibration comparisons.

That predictor comparison remains descriptive: replacing the forecasting configuration also changes residual and scenario distributions, so the observed outcome difference cannot be attributed causally to lower prediction error alone.

What operators should measure before replacing a routing system

For a navigation platform or delivery operator, the paper suggests a more demanding model-acceptance procedure.

First, distinguish four measurement layers: road-speed forecast error, path-time uncertainty coverage, selected-route rankings, and realized operational losses. Each layer answers a different question, and success at one does not guarantee success at the next.

Second, specify the business objective before choosing the routing interface. A service primarily concerned with arrival deadlines may need to compare estimated lateness probabilities against upper-bound and expected-time selection. An operator optimizing average travel time may prefer a different rule. The appropriate comparison depends on the actual loss function and service constraints.

Third, retain paired decision records whenever calibration or prediction components change. Recording the original route, replacement route, score margin, uncertainty adjustment, and realized outcome allows teams to distinguish a statistical improvement from an operationally consequential ranking change. Near-tied alternatives deserve particular inspection because the study finds that many switches occur at those boundaries.

These are Cognaptus interpretations for evaluation design, not deployment procedures experimentally validated by the paper. Their value lies in making acceptance criteria correspond to the decisions a predictive system actually controls.

What the offline benchmark cannot establish

The evidence has meaningful boundaries.

Both cities contribute only 14 test dates. Their road speeds come from processed public datasets whose upstream smoothing, missing-value treatment, and network restrictions cannot be fully reconstructed. Repeated route observations within those dates are dependent, so the large number of proxy decisions does not amount to an equally large number of independent traffic conditions.

There are also scenario-generation limitations. The formal Beijing Monte Carlo comparison did not satisfy prespecified convergence checks. The later E168-CDF diagnostic enumerated a fixed residual library, reducing conditional resampling uncertainty without demonstrating that the library represents future traffic. Chengdu’s formal residual support was limited to 15 vectors.

Sensitivity tests involving bootstrap blocks, residual dependence, alternative quantiles, travel-time budgets, and candidate-set sizes provide useful checks on the observed pattern. They do not supply independent external validation, and some expanded candidate-set tests change the feasible evaluation population.

Most importantly, candidate paths and budgets are frozen. The study does not model driver rerouting, congestion responses, traffic equilibrium, or live customer behavior. Its accompanying repository supports result-level verification rather than complete end-to-end reconstruction.

Consequently, the adverse C2 results should not be interpreted as proof that joint calibration is generally harmful. They establish a narrower finding about the evaluated calibration procedure, route selector, and offline tasks.

The acceptance criterion must include the decision

The central contribution is an empirical demonstration that statistical reliability can improve while downstream decision quality deteriorates.

The mechanism is concrete: path-specific uncertainty adjustments change relative route scores; some decisions reverse; and, in these frozen comparisons, harmful switches outnumber beneficial ones.

For operational AI, the resulting requirement is to test predictive improvements through the decisions they change. A calibrated bound can be statistically better and still be less suitable for a particular routing objective. Only by measuring both can an operator distinguish improved uncertainty representation from improved service performance.

Cognaptus: Automate the Present, Incubate the Future.