TL;DR for operators
A team running its own reasoning model has three familiar ways to improve accuracy: collect more labeled solutions, obtain a stronger teacher, or allow more computation per answer. LoOPSD1 investigates a fourth possibility: the model’s existing weights may provide better supervision when temporarily executed differently, and the improvement may survive after normal execution is restored.
A transformer normally processes information through its layers in a fixed sequence. LoOPSD constructs a temporary teacher by executing a selected block of existing layers twice, without adding new weights. Candidate execution routes are validated against the standard sequence, and a loop is selected only when it performs better. The student then generates its own responses and learns from the teacher’s token predictions along those responses. At deployment, the student still executes each original layer once.
On GSM8K, Qwen3-1.7B accuracy increases from 42.30% to 79.61%, a 37.31-percentage-point gain, under greedy decoding with a 400-token thinking cap and a separate 128-token answer cap (Section 5.1, Table 1). A matched-update ablation supports the contribution of repeated-layer supervision, while iterative training produces further improvements under the evaluated conditions.
For operators, the opportunity is better reasoning within a fixed serving budget without retaining the teacher’s extra layer executions. The decision is whether to invest in execution-route search and self-distillation rather than conventional supervision or larger inference budgets. The reported accuracy gains justify testing that alternative, but comprehensive training-compute and route-search costs remain unavailable. Improved benchmark accuracy is not yet demonstrated end-to-end cost efficiency.
A stronger teacher can come from repeating existing layers
A conventional transformer executes its layers in a fixed sequence. LoOPSD changes that sequence temporarily by selecting a contiguous interval of layers and executing it twice, using the same pretrained weights.
If the original sequence contains $L$ layers and the selected interval runs from $i$ through $j$, the alternative execution performs $L+(j-i+1)$ layer evaluations per generated token. No additional parameters are introduced. The intermediate representations change because the repeated layers process hidden states that have already passed through the same operations.
This makes the teacher potentially different from the standard model without requiring separately pretrained teacher weights. But a different execution route is not automatically a better one.
LoOPSD therefore searches and validates candidate routes. On GSM8K, the staged procedure initially evaluates candidate intervals using 128 labeled questions, screens survivors on another 872 questions, and compares the leading candidates against standard execution on 100 separate validation questions.
A nonstandard route is selected only when its validation accuracy strictly exceeds the standard route’s accuracy under identical prompts and generation budgets. If standard execution ties or wins, it remains the selected route.
This validation gate matters operationally. The route that improves predictions can vary with the checkpoint, task, and teacher prompt. The experiments include settings where no loop is selected. Repeated execution supplies candidate supervision; labeled validation determines whether that candidate is worth using.
The student learns from its own responses, not copied teacher solutions
Once LoOPSD selects an execution route, it freezes the resulting teacher for a distillation round. The student continues to use the original layer sequence.
Training proceeds through three connected operations:
- The current student generates responses to training questions using its standard execution path.
- The frozen teacher evaluates the next-token probabilities along those student-generated response prefixes.
- The student updates its parameters toward the teacher’s preferred token distribution.
This is on-policy self-distillation: supervision is applied where the student actually generates tokens, including prefixes associated with its mistakes, rather than exclusively along teacher-generated solutions.
The conceptual objective minimizes the student-to-teacher reverse Kullback–Leibler divergence:
The implementation does not calculate the complete vocabulary-level divergence directly at every training step. Instead, it uses a sampled-token policy-gradient surrogate. Tokens that the teacher favors relative to the student’s rollout policy receive positive update signals; disfavored tokens receive negative signals. The teacher-minus-rollout log-probability difference is clipped, while the importance ratio used in the surrogate remains unclipped (Section 3.2; Appendix A, Equations 4–6).
Reference answers serve a different purpose: they help select the teacher’s execution route. They are not supplied as target reasoning traces for each distillation response.
After a round of 200 student updates, LoOPSD can search again using the student’s updated weights. This iterative reconstruction makes the next teacher dependent on what the student has already learned. The process continues according to validation-based selection and continuation rules, with at most ten rounds.
The final artifact is a modified set of student parameters, not a model that must continue executing additional layers.
The benchmark gains survive the return to standard execution
The principal results compare original checkpoints, Elastic Reasoning, CRISP, and LoOPSD across mathematical and logical reasoning tasks. The table below selects final-checkpoint results from the main evaluations, all using greedy decoding with 400 thinking tokens and 128 answer tokens.
| Model and task | Original | LoOPSD | LoOPSD with concise teacher |
|---|---|---|---|
| Qwen3-1.7B · GSM8K | 42.30% | 79.61% | 79.68% |
| Qwen3-4B · GSM8K | 74.98% | 89.76% | 91.13% |
| Qwen3-8B · MATH-500 | 39.00% | 49.60% | 65.80% |
| Qwen3-8B · ProofWriter | 65.40% | 79.40% | 81.60% |
| Phi-4-reasoning-plus · GSM8K | 89.54% | 94.92% | 93.18% |
Source: Section 5.1, Tables 1–2. All entries are final rather than first-round accuracy. Task-specific training is used for the listed tasks.
The gains are not confined to the smallest checkpoint. On Qwen3-8B MATH-500, the concise-teacher variant reaches 65.80%, compared with 45.80% for CRISP. On Phi-4-reasoning-plus GSM8K, ordinary LoOPSD reaches 94.92%, above the 89.54% original checkpoint.
The concise-teacher extension changes the teacher’s instruction during route search and distillation while preserving the student’s original prompt. It can be effective, but it is not consistently superior to ordinary LoOPSD. Furthermore, validation sometimes retains standard execution under concise prompting, so the variant’s gains cannot always be attributed specifically to repeated layers.
Two ablations are particularly informative.
First, repeated-layer supervision contributes beyond ordinary self-distillation. On Qwen3-4B GSM8K with a 128-token thinking budget, looped-teacher training reaches 50.87% accuracy after 1,200 student updates, compared with 26.38% for standard-path self-distillation using matched prompts, objectives, and update schedules (Section 6.2, Figure 5(a)). That 24.49-point difference makes the execution-derived teacher a supported contributor rather than merely an unusual training configuration.
Second, improvements continue beyond the first round. Along the same six-round LoOPSD trajectory, accuracy increases from 33.81% after the first round to 50.87% after the sixth. This supports the value of continued search and learning under the evaluated schedule, although later checkpoints also incorporate additional updates and computation. It does not establish that every additional round necessarily improves performance.
Better answers can also be shorter, but token counts are not total costs
For Qwen3-4B on GSM8K, ordinary LoOPSD increases accuracy from 74.98% to 89.76% while reducing average total generated tokens from 385.04 to 299.98 (Section 5.3, Figure 4).
That is approximately 22.1% fewer generated tokens alongside a 14.78-point accuracy improvement.
The pairing matters: shorter generation is more valuable when the same responses also become more accurate. However, the length result measures generated tokens, not actual serving latency or training expense. It excludes the computations needed to search execution routes and train the student.
The secondary evaluations add useful boundaries. GSM8K-trained students improve accuracy on SVAMP and GSM-Plus without target-task training. For Qwen3-4B, ordinary LoOPSD reaches 92.67% on the 300-question SVAMP transfer set versus 87.33% for the original model, and 74.20% on the fixed 1,000-question GSM-Plus subset versus 56.40% (Appendix C, Table 4). These are meaningful cross-dataset results within mathematical reasoning, not evidence of transfer to arbitrary business tasks.
Generation-temperature sensitivity tests generally preserve the improvements. Layerwise ProofWriter readouts also show stronger correct-answer predictions concentrated in later layers after distillation. Those diagnostics are consistent with a changing internal prediction process, but they do not establish a complete causal account of how reasoning improves.
The operational test is accuracy gained per complete training budget
For teams adapting self-hosted reasoning models, the immediate decision is whether to experiment with execution-derived supervision alongside conventional fine-tuning or stronger external teachers.
The paper’s label-efficiency experiment supplies one reason to investigate. With 200 total labeled examples, including 100 validation examples, Qwen3-1.7B reaches 78.01% GSM8K accuracy under LoOPSD versus 43.90% for GRPO after 200 updates (Section 6.1, Table 3).
This comparison matches labeled-example totals and student updates, but not all resources. LoOPSD additionally uses a question-only distillation pool and performs teacher-route search. Consequently, the result concerns utilization of labeled feedback under the specified protocols, not total computational efficiency.
Cognaptus inference: A team with a verified-answer dataset and a larger pool of unlabeled questions could test whether concentrating labels on teacher selection provides stronger training signals than spending the same annotation budget on direct optimization. The evaluation should compare final standard-path accuracy, generated-token counts, training FLOPs, route-search overhead, actual inference latency, and transfer performance on held-out application tasks.
A useful experimental design would compare the original checkpoint, conventional self-distillation, and LoOPSD under a shared deployment configuration. Separate reporting of first-round and final checkpoints would expose the marginal value of further training. Testing the selected route against standard execution on an independent validation set would also reduce the risk of adopting a loop that only performed well during candidate screening.
The missing comparison is end-to-end economics
LoOPSD offers credible benchmark-specific evidence, but three boundaries limit immediate production conclusions.
The evaluation concentrates on mathematical and logical reasoning benchmarks. It does not establish equivalent gains for coding, multimodal applications, business workflows, or safety-critical decisions. The reported training uses a single seed, without comprehensive multi-seed uncertainty estimates for the main results. Route selection also depends on finite validation sets whose stability under alternative samples is not fully characterized.
Most consequentially, comprehensive teacher-search FLOPs, training cost, energy use, and wall-clock comparisons are unavailable. A standard-execution student avoids retaining the teacher’s repeated-layer inference path, but that architectural property alone does not establish a favorable total cost of ownership.
The result changes how teams can construct supervision: a temporary change in computation can produce a persistent improvement in model parameters. Whether that improvement justifies the additional training process remains a measurable operational question.
For organizations able to run controlled model-adaptation experiments, LoOPSD merits evaluation as an alternative to acquiring stronger teachers or permanently increasing inference computation. The adoption threshold should be improved performance on representative tasks at an acceptable combined training and serving cost—not benchmark accuracy in isolation.
Cognaptus: Automate the Present, Incubate the Future.
-
Wei Fan and Qingyue Yang and Chao-Hong Tan and Qian Chen and Wen Wang and Xiangang Li and Kejiang Chen and Weiming Zhang and Nenghai Yu (2026). LoOPSD: On-Policy Self-Distillation with Looped Teachers. arXiv:2610.18162. https://arxiv.org/abs/2610.18162 ↩︎