When the Simulator Becomes the Curriculum

TL;DR for operators

A simulator can generate effectively unlimited trajectories. That does not automatically make those trajectories useful training data for a reasoning model.

Sim2Reason turns simulated mechanics into questions with answers that can be checked automatically, then uses those questions for reinforcement-learning post-training. The reported transfer is substantial enough to matter operationally: on Qwen2.5-32B, International Physics Olympiad mechanics accuracy rose from 19.8% to 25.2%. By contrast, supervised fine-tuning on 200,000 teacher-generated trajectories lowered the same score to 15.9%.

The relevant lesson is therefore not “generate more synthetic data.” It is to build a pipeline in which synthetic tasks require the physical mechanism being simulated, unstable or shortcut-solvable examples are removed, and improvements are still tested against real-domain problems. Simulators can expand the supply of training and evaluation evidence; they do not remove the need for external validation.

More demonstrations did not produce better transfer

Suppose an engineering team already has a simulator that encodes years of domain knowledge. It can generate more state trajectories than experts could ever annotate manually. The harder question is whether those trajectories can teach a language model to solve unfamiliar problems rather than simply imitate patterns produced by the simulator.

That distinction becomes concrete in Solving Physics Olympiad via Reinforcement Learning on Physics Simulators.¹ In the paper’s Qwen2.5-32B experiment, reinforcement learning on simulator-generated questions moved IPhO mechanics accuracy from 19.8% to 25.2%. Supervised fine-tuning, despite using 200,000 rejection-sampled trajectories from GPT-4, o3, and o4-mini, reduced accuracy to 15.9%.

This is a comparison of training strategies in the paper’s specific setup, not evidence that SFT generally damages physics reasoning. But it changes the engineering question. Once correct answers can be verified automatically, the scarce resource may no longer be teacher-written reasoning demonstrations. It may be the design of environments and tasks that supply useful outcome feedback.

The simulator is a supervision engine, not an inference-time solver

Sim2Reason does not ask the evaluated model to invoke MuJoCo when answering an olympiad problem. The simulator sits upstream.

The pipeline procedurally constructs physically meaningful scenes, executes them in MuJoCo, records quantities such as position, velocity, force, momentum, energy and contact forces, and then converts those traces into natural-language questions. It supports forward numeric questions, reverse parameter-inference questions and symbolic questions.

That transformation matters because a raw simulation trajectory answers a forward question: given this system, what happens next? Physics problems frequently demand something else: recover an unknown parameter, relate quantities symbolically or reason about a situation not reproduced exactly during training.

The resulting answers are automatically checkable. During reinforcement learning, a response receives positive reward when its final answer is within 5% relative error of the simulator value. This makes reinforcement learning from verifiable rewards, or RLVR, possible without supplying a canonical teacher reasoning trace.

For an organization with an established scientific or engineering simulator, the analogous opportunity is to treat executable domain knowledge as a source of renewable supervision. That requires more than exposing an LLM to simulator logs. The difficult layer is converting simulator state into tasks that exercise the reasoning capability the downstream product actually needs.

Synthetic questions have to depend on the system they describe

Synthetic-data pipelines have an uncomfortable failure mode: a generated question can look physically rich while its answer barely depends on the supposedly relevant physics.

Sim2Reason explicitly tests for this. It simplifies a generated scene—for example by removing entities or replacing joints with rigid components—and re-runs the simulation. If the answer remains unchanged, the question is considered shortcut-solvable and discarded. About 15% of generated pairs are removed this way.

The ablation indicates that this is not merely data hygiene. On Qwen2.5-3B, the baseline IPhO score is 5.68%. Reinforcement learning with unfiltered synthetic questions reaches 7.14%; using the filtered set reaches 13.15%.

A second ablation tests question format. From the same 5.68% baseline, reverse questions reach 5.84%, symbolic questions 7.46%, and numeric questions 13.15%. Numeric QA is consequently the default training distribution in the reported system.

For operators, this is a more demanding standard for synthetic-data quality than plausibility. A task should change when the causal structure that supposedly determines its answer changes.

Transfer appears across model scales, not just inside the simulator

The main evidence is the zero-shot evaluation on real physics problems after synthetic-only post-training.

IPhO mechanics accuracy increases at every reported model scale: Qwen2.5-3B moves from 5.68% to 13.15%, 7B from 10.7% to 15.1%, 14B from 16.07% to 20.45%, 32B from 19.8% to 25.2%, and Qwen3-30B from 35.6% to 40.0%.

For Qwen2.5-32B, gains also appear on JEEBench, PHYSICS and OlympiadBench. JEEBench rises particularly sharply, from 34.38% to 52.28%. Smaller improvements appear on the out-of-domain mathematics evaluations AIME 2025 and MATH 500.

A matched Qwen2.5-3B comparison adds another useful signal. Synthetic mechanics RL reaches 13.15% IPhO accuracy, versus 9.98% after RL on the 17,000-problem mathematics-focused DAPO-17K dataset. A mixed DAPO-plus-synthetic condition reaches 10.35%.

The paper therefore supports a narrower conclusion than “synthetic data beats real data.” The matched alternative is mathematics data, not a comparable public physics post-training dataset. What the experiment does suggest is that domain alignment can outweigh raw dataset size when the target capability is specialized.

The synthetic environment can also become a diagnostic layer

Training is only one use of the generated questions.

Across the evaluated models and runs, accuracy on Sim2Reason synthetic questions has a Spearman correlation of 0.79 with IPhO mechanics accuracy. That makes simulator QA potentially useful as a cheaper, reproducible proxy for tracking physical-reasoning capability.

For model-evaluation teams, the attractive workflow is not to replace expert benchmarks with simulator scores. It is to run synthetic diagnostics frequently, then reserve scarce real-domain evaluations for checkpoints, regressions and release decisions.

The correlation is evidence that the proxy contains signal. It is not evidence that synthetic and real evaluation are interchangeable.

Build the generator around the simulator’s abstraction boundary

Scene generation itself introduces another operational issue. Asking an LLM to emit raw simulator code is brittle.

Sim2Reason instead defines a compositional domain-specific language for scenes, entities and bodies, constraining randomization to physically meaningful parameters before compiling scenes into MuJoCo XML. In a small three-problem exploratory extension study, direct raw XML generation succeeds in one of three cases, whereas the DSL route succeeds in all three with minor corrections.

Three cases are far too few for a general reliability claim. They nevertheless illustrate a design principle worth testing in engineering systems: where a mature simulator has a complex low-level interface, an intermediate domain abstraction can expose the decisions the model should make while hiding implementation details it does not need to control.

Real benchmarks remain the deployment boundary

The evidence remains concentrated in classical mechanics. Simulator outputs are approximations, and their modeling assumptions can become assumptions in the training signal. Broader areas such as electromagnetism and thermodynamics are not established by these results.

Several evaluations also narrow the input modality: some benchmarks use text-only mechanics subsets, while diagram-containing IPhO and HCV questions receive GPT-4o-generated figure captions. Qwen3-30B receives only 100 reinforcement-learning steps rather than the 200 used for Qwen2.5 because its much longer responses increase training cost.

Cognaptus therefore sees the business opportunity as an infrastructure decision for teams that already possess trusted executable models of a domain. Instead of repeatedly purchasing or authoring static QA, they can investigate whether the simulator can generate verifiable tasks, whether those tasks survive causal shortcut tests, and whether learning from them improves the real decisions the model will face.

The simulator can make supervision renewable. It cannot certify its own transfer.

Cognaptus: Automate the Present, Incubate the Future.