Backpropagate Less, Match More

TL;DR for operators

Elastic Forcing changes where video-model post-training spends memory.

Instead of keeping a diffusion teacher and an online fake-score model resident during distributional training, it compares generated videos directly with reference videos in frozen representation spaces. The reported 14B setup completes 80 post-training updates on eight H200 GPUs in 23.2 hours, while the comparable Self-Forcing configuration is reported to run out of memory on eight GPUs.

The more interesting design choice is not simply “use fewer samples.” Elastic Forcing can generate a relatively large population of videos to define the distribution-matching objective, calculate the feature-space update signal across that population, and then replay only a sampled subset through the generator for backpropagation. In one ablation, increasing the distribution batch from 128 to 256 while holding the differentiated batch at 128 raises VBench Total from 84.09 to 84.25 and Quality from 84.77 to 85.06.

For teams adapting large streaming video generators, the paper points toward two separable resource decisions: how many outputs should inform the target distribution, and how many outputs can affordably carry gradients. It also makes reference data itself a configurable supervision target. The boundary is equally important: larger rollout populations still cost forward compute, generated–generated kernel comparisons scale quadratically with batch size, and the evidence remains tied to the tested Wan models, representation encoders, and adaptation tasks.

The expensive part is not generating every candidate

A large generative model can often produce more candidate outputs than a training system can afford to retain for reverse-mode differentiation. That distinction matters because distributional objectives benefit from observing a population of outputs, while backpropagating through that entire population can dominate memory.

Existing few-step autoregressive video post-training commonly adds another burden: score-based supervision. A diffusion teacher and an online model of the generator’s score distribution may both participate in training. More supervision machinery therefore means more state competing for the same accelerator memory.

This is the systems problem addressed by From Scores to Samples: Elastic Forcing for Autoregressive Video Generation.¹ The paper asks whether the target distribution can instead be defined directly by reference videos, and whether the population used to estimate that distributional mismatch can be larger than the population through which gradients are propagated.

Replace score supervision with reference distributions

Elastic Forcing retains the autoregressive rollout procedure used by Self-Forcing but replaces Distribution Matching Distillation with maximum mean discrepancy, or MMD.

In practical terms, MMD asks whether generated and reference samples occupy similar distributions in a chosen representation space. The paper freezes video encoders including V-JEPA 2 and VideoMAE v2, maps both generated and reference videos into those spaces, and minimizes their distributional discrepancy.

The central objective is:

This changes the supervision architecture. The desired target is represented by samples rather than by a target-specific diffusion score model.

The matched 1.3B comparison gives the cleanest evidence for whether that substitution works. With the same architecture, ODE initialization, rollout procedure, and 17 FPS inference speed, the two-encoder Elastic Forcing configuration reaches 84.64 VBench Total versus 83.80 for Self-Forcing. Quality rises from 84.59 to 85.43 and Semantic from 80.64 to 81.48.

These are incremental benchmark gains rather than evidence of a new quality regime. Their relevance is that the simpler supervision stack does not require accepting a measured quality loss in the matched experiment.

Reference estimation becomes a bias–variance problem

Direct sample matching creates another problem. A small minibatch of reference videos provides a noisy estimate of the target distribution, but comparing every generated sample against the full reference collection would be expensive.

Elastic Forcing handles this with a hybrid estimator.

The Nyström component builds a persistent finite-rank approximation from landmarks over reference features. It supplies reusable global information with relatively low variance but introduces approximation bias. Monte Carlo sampling uses fresh reference minibatches. It is unbiased for the empirical reference distribution but noisier.

The method mixes the two:

The estimator ablation is best read as component evidence rather than as a second headline result. The hybrid configuration records VBench Total 84.64 and VLM 2.07, compared with 84.59 and 1.97 for Nyström alone, and 84.06 and 1.97 for Monte Carlo alone. Nyström alone is fractionally higher on VBench Quality, at 85.47 versus 85.43.

The result supports the paper’s intended bias–variance trade-off in this configuration. It does not establish a universally optimal mixture, and the paper’s analytical optimal coefficient is an oracle expression rather than the coefficient claimed to be estimated exactly during training.

More rollouts can shape the objective than carry gradients

The most transferable systems idea is the separation between distribution estimation and differentiation.

Elastic Forcing first generates a distribution batch of $B$ fresh on-policy rollouts. It computes the MMD interactions across this population and obtains feature-space cotangents without retaining every generator computation graph. It then samples only $b$ rollouts, reconstructs them through replay, and backpropagates their cached feature-space signals.

The gradient contributions are rescaled by $B/b$. Under the paper’s replay and sampling conditions, that sampled Jacobian sum is conditionally unbiased for the gradient of the evaluated finite-batch objective.

The distinction is narrow but important: the claim concerns the gradient of that evaluated batch and reference estimate. It is not a claim that the procedure exactly recovers the population MMD gradient.

The batch ablation shows why the separation can matter. Holding $b=128$ fixed while increasing $B$ from 128 to 256 moves VBench Total from 84.09 to 84.25 and Quality from 84.77 to 85.06. More outputs can therefore influence the distributional signal without requiring the backward batch to grow with them.

For infrastructure planning, this suggests that “batch size” is too coarse a resource variable. A training system may want a relatively broad forward population for statistical estimation and a narrower backward population for differentiation.

The 14B result is mainly a memory result

At 14B scale, Elastic Forcing post-trains Wan2.1-T2V-14B for 80 updates on eight H200 GPUs in 23.2 hours, equal to 185.6 GPU-hours. The paper reports that Self-Forcing at 14B runs out of memory on eight GPUs.

This does not mean Elastic Forcing makes large-model development cheap. The figure covers post-training only. It excludes foundation-model pretraining and the earlier ODE initialization.

What the experiment directly supports is more specific: removing auxiliary score networks can move a post-training workload from infeasible to feasible within the tested eight-GPU memory envelope.

The larger model also scores 4.217 on the paper’s VLM total versus 4.115 for Krea Realtime, while blinded human ratings average 3.354 versus 3.054. Those complementary evaluations matter because the paper explicitly notes that VBench does not consistently capture larger-model perceptual differences.

Reference data becomes part of product configuration

Replacing a teacher-defined target with reference samples creates another possible use: customization.

The paper adapts the 1.3B generator using reference collections for monochrome appearance, the Nailong character, and a panoramic/spatial appearance target, without first constructing a target-specific diffusion teacher. The largest task-specific change appears in the spatial experiment, where the score rises from 5.000 for Self-Forcing to 8.544 for Elastic Forcing.

Cognaptus inference: for teams building product-specific visual behavior, a reference-defined objective could shorten the path between collecting acceptable examples and adapting the generator. This is most relevant when the desired target is easier to specify as a video collection than as another trained generative teacher.

That inference should remain bounded. The experiments demonstrate selected appearance, identity, and spatial-presentation targets. They do not establish unrestricted concept learning, correct 3D geometry, or general physical reasoning.

The compute has moved, not disappeared

Elastic Forcing should not be read as a method that becomes efficient simply by using fewer samples.

All $B$ distribution rollouts still require forward generation. Generated–generated kernel interactions remain quadratic in $B$. The Nyström approximation introduces finite-rank and regularization error, while subsampling differentiated rollouts adds gradient variance. The method also relies on frozen representation encoders, and feature-distribution matching cannot penalize information those encoders fail to preserve.

Encoder choice already shows that this dependency is material. V-JEPA 2 plus VideoMAE produces the strongest reported VBench aggregate, while adding DINOv3 lowers VBench Total from 84.64 to 83.81 but raises the comparative VLM score from 3.03 to 3.63. The supervision geometry therefore changes what the optimization rewards.

The practical conclusion is narrower than “sample matching is cheaper.” Elastic Forcing reallocates training resources: less memory for auxiliary score models, more freedom to use reference data as the target, and a deliberate split between the number of samples used for distribution estimation and the number differentiated through the generator.

For large generative systems, that separation may be the lasting design idea. Statistical evidence and gradient computation do not necessarily need to consume memory at the same scale.

Cognaptus: Automate the Present, Incubate the Future.