The One-Step Mirage: Stress-Testing World Models Beyond the Next Frame
TL;DR for operators A world model can be reasonably accurate when predicting the next state and still become unreliable once its own predictions are fed back repeatedly. That difference is operationally relevant because planning and control systems often depend on trajectories, not isolated one-step estimates. Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization1 tests this problem by training models on a narrow range of gravity values and evaluating them across much wider regimes. SG-JEPA jointly trains its visual representation and temporal predictor using several recursively generated future latents. Its largest advantages generally appear where errors have more opportunity to compound: longer forecast horizons and gravity values outside the training distribution. ...