A Better Forecast Can Choose a Worse Plan

TL;DR for operators

An inventory manager can obtain accurate forecasts of stock and backlog yet still choose an inferior replenishment schedule. Forecasting future conditions, predicting how those conditions change under alternative actions, and ranking the available plans are different capabilities.

ActTime: Action-Conditioned World Modeling for Time Series¹ addresses this distinction by learning controlled dynamics from historical action-outcome trajectories. It then simulates alternative futures from the same observed history, without requiring paired counterfactual outcomes during training.

Across four simulated operational domains and eight evaluated methods, ActTime leads in 12 of 16 domain-task settings. More revealingly, 17.8% of pairwise method rankings reverse between factual forecasting and intervention-effect prediction, rising to 28.4% between intervention-effect prediction and decision-ranking quality.

For operators selecting forecasting or optimization systems, historical forecast accuracy is an incomplete acceptance criterion. Candidate-action rankings and realized operating costs need their own tests. The evidence supports this distinction within controlled simulators; it does not establish performance in physical deployments.

Why forecasting performance can mislead plan selection

Consider an inventory planner choosing among replenishment schedules for a retailer, distributor, and factory. Each schedule produces a different trajectory for stock levels, backlog, and costs.

A conventional forecasting test asks how accurately a model predicts the trajectory that actually occurred. The planning problem is broader: would changing the replenishment schedule reduce backlog sufficiently to justify its inventory cost?

Even a model that predicts the difference between two schedules relatively accurately may rank the schedules incorrectly if its remaining errors change their estimated costs.

The ActTime study quantifies this separation across eight methods. In its cross-level alignment analysis, pairwise model-ordering reversals average 17.8% when moving from factual prediction error to intervention-effect error. They average 28.4% when moving from intervention-effect error to decision-ranking quality (Section 4.3; Appendix B.2, Table 12).

These percentages describe reversals in the relative performance of competing models, not the proportion of individual operational decisions that fail. The associated macro Spearman correlations fall from 0.763 for prediction versus intervention to 0.514 for intervention versus decision ranking.

An organization could therefore select one model using forecasting accuracy and a different model using the quality of the decisions its predictions support.

ActTime learns to simulate actions without counterfactual training labels

An operational planner needs to examine multiple possible futures from a common starting condition. ActTime provides a learned simulator for this purpose.

Its inputs include historical observations, executed historical actions, and a proposed sequence of future actions. Its outputs are the predicted future target trajectories under that proposed sequence.

The architecture has four relevant components.

First, variable-specific latent states represent different observed quantities rather than forcing all variables into one shared state. Cross-variable attention allows those representations to interact.

Second, a shared action-conditioned transition updates the states using their previous values, cross-variable information, current action levels, and changes in actions. This is the component that makes predicted dynamics responsive to the controls being applied.

Third, observation correction adjusts the propagated states against actual measurements during historical filtering. It prevents the inferred starting state from depending entirely on accumulated model predictions.

Fourth, a frozen pretrained EIDOS temporal backbone supplies historical features through a separately trained residual adapter. The adapter adjusts the terminal history state; it does not replace or learn the controlled transition.

After processing the observed history, ActTime repeatedly applies the learned transition under each candidate future action sequence. Future observations are unavailable during these open-loop rollouts, so predictions depend on the inferred starting state, proposed actions, and learned dynamics.

Training is deliberately narrower than the final application. ActTime minimizes quantile prediction loss on realized trajectories only, first training its controlled dynamics and prediction head, then its temporal-prior adapter. Checkpoints are selected using factual validation MAE.

Neither paired alternative-action outcomes nor decision-specific training rewards are used.

This matters because organizations often possess historical actions and subsequent measurements without records of what would have happened under every unchosen action. ActTime shows a way to construct alternative-action simulations from such factual supervision, provided the historical data contain enough variation to learn the relevant responses.

It does not follow that these simulations identify causal effects under hidden confounding.

Four simulated systems expose different strengths

The researchers evaluate ActTime against seven alternatives: Ridge-VARX, action-conditioned DLinear, PatchTST, iTransformer, TFT, RSSM, and MTS3.

The four environments represent battery scheduling in CityLearn, traffic-signal control in SUMO Cologne1, greenhouse climate control in GreenLight, and multistage replenishment in an inventory simulator.

Each model faces four evaluation settings:

1. Factual forecasting: How accurately does it predict outcomes under realized actions?
2. Intervention-effect prediction: How accurately does it predict changes caused by alternative actions relative to a reference plan?
3. Offline decision making: How much excess simulator cost results from selecting a candidate using model predictions rather than the simulator-best candidate?
4. Closed-loop planning: What operating objective is realized when the model guides repeated planning and execution?

For intervention tests, candidate plans start from matched simulator states and share matched exogenous conditions. This supplies a controlled comparison that ordinary historical test sets cannot provide.

The main results (Section 4.2, Table 2) show both ActTime’s advantages and their limits.

ActTime’s average rank is 1.50, compared with 2.25 for TFT, the strongest baseline by average rank.

Its factual MAE improves over the strongest baseline by 12.2% on SUMO, 25.2% on GreenLight, and 20.4% on Inventory. But its superiority in forecasting does not extend uniformly to intervention effects.

On SUMO, for example, ActTime has lower factual MAE than TFT (0.1076 versus 0.1226), while TFT has lower intervention Effect NMAE (0.4184 versus 0.4825).

The comparison is particularly informative because it occurs within one simulated system. A better observed-trajectory forecast is not automatically a better estimate of what changes when the traffic controls change.

Correctly ranking candidate plans is a separate achievement

ActTime also performs well on the ordering of candidate actions themselves.

For each held-out history, the researchers compare predicted candidate costs with simulator-derived costs. Under the predefined primary objective, ActTime achieves the highest mean per-history Spearman correlation in every domain.

Its four-domain average is 0.8661, versus 0.8154 for TFT, the strongest baseline on that aggregate measure (Section 4.4, Table 3).

The appendix extends this evaluation across 12 domain-objective combinations. ActTime leads on three macro-averaged statistics: Spearman correlation (0.8270), Kendall tau-b (0.7272), and pairwise ordering accuracy (0.8624).

This is a robustness test of the candidate-ranking result across additional objectives and ranking measures, rather than a new demonstration of real-world control.

Ranking fidelity also differs from offline regret. A model can order many candidate pairs correctly yet make a consequential error among the best few plans. Offline regret instead measures the excess realized simulator cost of the selected plan, normalized against the gap between random-reference and oracle selection.

Both quantities are relevant when predicted trajectories are used to choose actions, but neither substitutes for observing closed-loop performance.

Action conditioning matters more consistently than pretraining

The structural ablations help distinguish which architectural choices contribute to performance (Section 4.5, Table 4).

Removing action conditioning raises the four-domain macro-average ratio of intervention error to 6.393 times that of the matched full model. The corresponding offline-regret ratio rises to 16.349.

Replacing variable-specific states with a shared state produces intervention-error and offline-regret ratios of 5.367 and 4.360. Removing observation correction increases those two ratios to 2.132 and 2.215.

These are normalized matched-ablation comparisons, not the final-checkpoint scores from the main benchmark. Their purpose is to test structural contributions, not to provide another set of production performance estimates.

The pretrained temporal prior has a less uniform contribution.

In the exactly paired comparison of models with and without the EIDOS adapter (Appendix B.4, Table 15), the adapter improves the primary closed-loop objective in all four environments. However, factual error, intervention error, and offline regret improve in some domains and deteriorate in others.

For instance, SUMO intervention Effect NMAE is 0.4825 with the prior versus 0.4355 without it, although the closed-loop objective improves from 0.3097 to 0.3036.

The evidence therefore supports a narrower architectural conclusion: explicit controlled dynamics and state correction are central to the tested performance, while pretrained temporal features can assist planning without uniformly improving intermediate prediction metrics.

What operational buyers should require before deployment

Cognaptus inference: The study suggests a staged evaluation process for teams purchasing or building AI-assisted operational planners.

An inventory operation considering automated replenishment should first measure historical forecast error, then test whether the model captures the consequences of feasible changes in replenishment quantities. It should separately compare the resulting candidate rankings against a trusted evaluation environment and measure realized stock, backlog, and service costs during controlled deployment testing.

The corresponding decision variables differ by operation: charge and discharge schedules for energy managers, signal-phase sequences for traffic controllers, and heating, ventilation, or CO2 settings for greenhouse operators.

The procurement implication is not that every organization needs ActTime specifically. It is that model selection needs to reflect the action-selection task, including its feasible controls, objective function, and operating constraints.

Three evidence boundaries matter.

First, all four environments are simulated. Their matched alternative-action outcomes support simulator-relative intervention comparisons, not causal identification from observational data or guarantees of physical-system performance.

Second, candidate actions remain within the simulators’ feasible ranges. Historical action coverage is necessary for learning useful responses; substantially unsupported interventions remain untested.

Third, closed-loop evaluation uses finite-horizon model predictive control with cross-entropy-method optimization, primarily relying on median point predictions. Although ActTime produces predictive quantiles, calibrated uncertainty under changed action distributions and risk-sensitive planning are not established.

Those boundaries are material for any operator considering automated control over assets, inventory, or service levels.

The model that forecasts best need not decide best

ActTime’s most consequential result is not simply its 12 benchmark wins. It is the measured separation between predicting observed trajectories, estimating alternative-action effects, and choosing plans with favorable realized costs.

Its learned dynamics show that factual-trajectory supervision can support useful alternative-action simulation in the tested systems. Its evaluation shows why forecast accuracy alone cannot certify that simulation for decision making.

For operational AI, the acceptance test ultimately needs to reach the decision the system is expected to make—and the cost of executing it.

Cognaptus: Automate the Present, Incubate the Future.