TL;DR for operators
Access to a stronger AI model’s solutions does not guarantee that copying those solutions will improve a smaller model. In one experiment, supervised fine-tuning reduced a 0.6B-parameter student’s mathematical reasoning accuracy from 10.8% to 2.8%. An alternative training procedure raised it to 12.5%.
Bridged Self-Distillation (BSD) uses external-model responses as privileged training context rather than outputs the student must imitate. A frozen copy of the student reads the external response and provides token-level guidance on the student’s own attempts. An automatic verifier still determines whether those attempts deserve reinforcement or penalties.
Across 12 teacher–student configurations, BSD improves average accuracy over the original student in every case and achieves the highest or tied result among distillation methods in 11. Its interface works with shared-tokenizer models, different-tokenizer models, and text-only external APIs.
For developers of compact reasoning models, this reduces the need for specialized teacher-probability interfaces. It does not eliminate verification requirements, teacher-response costs, or deployment-specific evaluation. Improvements in average correctness also do not consistently improve the chance that at least one of several attempts succeeds.
When better examples make a smaller model worse
An AI development team has access to a powerful external model and wants a smaller internal model to solve similar problems independently. One approach is to collect the stronger model’s worked solutions and fine-tune the smaller model to reproduce them.
That approach assumes the teacher’s output is suitable for the student’s capabilities. A smaller model may struggle to reproduce a stronger model’s reasoning structure, or it may adopt response patterns that interfere with its existing behavior.
The consequences appear clearly in Bridged Self-Distillation for On-Policy Learning by Chen and colleagues.1 When a Qwen3-0.6B student was trained on responses from a closed-API teacher, supervised fine-tuning (SFT) reduced its average mathematical reasoning accuracy from 10.8% to 2.8%. BSD instead achieved 12.5% using the same student backbone and training questions.
The researchers also observed substantial changes in response length under some SFT configurations, particularly when students imitated the closed-API teacher’s comparatively short responses.
These comparisons do not prove that direct imitation must fail: the training procedures also differ in optimization details. They do establish a consequential design problem. Obtaining a capable teacher’s answers and transferring its useful reasoning behavior are different objectives.
Conventional on-policy distillation addresses part of this problem by training on solutions generated by the student itself. But many implementations require the teacher’s token probabilities, which a commercial API may not provide. Different tokenizers create another integration problem because the teacher and student may divide identical text into different tokens.
BSD addresses both constraints by changing how external knowledge enters the training process.
Teacher text becomes context, not something to imitate
BSD separates the source of reasoning information from the model that supplies token probabilities.
Before training, an external teacher generates a textual solution for each question. During training, the student receives the original question without that privileged solution and generates its own attempts, or rollouts.
A frozen copy of the student then receives both the question and the external teacher’s solution. This copy, called the privileged self-teacher, evaluates the exact tokens the unassisted student generated.
The distinction matters: the external model supplies text, while both sets of token probabilities come from versions of the student. They therefore use the same vocabulary, even when the external teacher has a completely different tokenizer.
The core guidance signal is:
Here, $p_t$ is the student’s probability of its generated token without the teacher’s trace, and $q_t$ is the frozen self-teacher’s probability of that same token after reading the trace.
A positive difference means the external information makes that token more plausible to the self-teacher. A negative difference means it makes the token less plausible.
The self-teacher is synchronized with the student every ten training updates. It is not another independent large model, and it does not generate replacement solutions for the student’s rollouts.
This arrangement enables training from external text without accessing external logits. It also allows visible responses from closed APIs to serve as privileged context when private reasoning traces are unavailable.
The verifier determines correctness; the trace determines penalty strength
Having token-level guidance is not equivalent to knowing which tokens should be rewarded.
BSD retains an automatic correctness verifier. It checks the student’s completed solutions and uses group-relative feedback, following Group Relative Policy Optimization (GRPO), to determine the direction of learning.
The external trace then modifies the strength of penalties on negatively advantaged rollouts.
If the self-teacher considers a token less plausible after reading the reference, BSD penalizes that token more strongly when the student’s attempt is unsuccessful. If the reference makes the token more plausible, BSD reduces the penalty. The multipliers are positive and clipped, so they do not reverse the verifier’s update direction.
Successful rollouts keep their original advantages. BSD does not automatically reward every token that the reference appears to favor.
The component ablations show why this separation matters.
| Training variant | Avg@4 |
|---|---|
| Original Qwen3-4B | 59.2% |
| GRPO without external-trace guidance | 60.6% |
| BSD with brief hints | 61.7% |
| BSD with structured guides | 62.5% |
| BSD with full reasoning traces | 63.9% |
| BSD also reweighting successful rollouts | 61.4% |
| Verifier-free OPSD | 59.7% |
Source: Section 6.1 and Appendix D, Table 11. Qwen3-4B student with Qwen3-32B teacher; development-selected checkpoints.
These are targeted ablations, not the full 12-pair comparison. They indicate that complete privileged reasoning, verifier feedback, and selective reweighting contribute to the observed performance in this configuration.
In particular, extending guidance-based weighting to successful rollouts reduces accuracy by 2.5 percentage points. Removing verifier feedback produces an even weaker result, with declining training performance and shorter responses.
The paper’s formal analysis offers a compatible interpretation: under a small-guidance approximation, BSD behaves like ordinary GRPO plus an outcome-weighted, token-level distillation correction restricted to unsuccessful rollouts. That first-order decomposition ignores locally inactive clipping and is not a guarantee of convergence or correct token-level credit assignment.
The benchmark gains cross three teacher-access conditions
The main comparison tests four Qwen3 student backbones against three teachers: Qwen3-32B with a shared tokenizer, Nemotron-3-30B-A3B with a different tokenizer, and a text-only GPT-5.6 Luna API model.
Each teacher–student pair uses 2,400 mathematical training problems. Checkpoints are selected using a separate development set, and performance is evaluated on AIME 2024, AIME 2025, and HMMT 2025, with four generated answers per problem.
BSD improves Avg@4 over the original student in all 12 pairs, by 1.2 to 4.7 percentage points. It achieves the highest or tied average among the tested distillation methods in 11 pairs.
For Qwen3-4B, the results illustrate how the comparisons change with teacher access:
| Teacher access | Base | SFT | Access-specific OPD | BSD |
|---|---|---|---|---|
| Shared tokenizer | 59.2% | 60.6% | 60.3% | 63.9% |
| Different tokenizer | 59.2% | 63.6% | 59.7% | 63.6% |
| Closed API | 59.2% | 57.2% | 59.7% | 61.9% |
Source: Section 5.1, Table 1. The OPD baselines are vanilla OPD, xOPD, and rubric-based OPD, respectively.
The broader pattern favors BSD, but not without exceptions. For Qwen3-0.6B with the different-tokenizer teacher, specialized xOPD reaches 14.2%, exceeding BSD’s 12.8%.
There is also an important distinction between evaluation objectives. Avg@4 measures average correctness across four sampled answers. Pass@4 measures whether at least one of those attempts solves the problem. In the shared-tokenizer Qwen3-4B comparison, Avg@4 increases from 59.2% to 63.9%, while average Pass@4 changes from 74.4% to 73.3%.
A system that produces one answer per request and one that generates several candidates for selection may therefore rank training methods differently.
The paper also investigates where guidance concentrates. On unsuccessful rollouts, the Spearman correlation between absolute token-guidance magnitude and student token entropy ranges from 0.83 to 0.90. More uncertain tokens tend to receive stronger guidance, but the analysis does not establish that those particular tokens caused the accuracy improvements.
Integrating external teachers still has a compute and verification cost
For organizations adapting compact internal models with external AI services, Cognaptus infers that BSD changes the integration decision.
A team with a text-only teacher API could collect reference solutions once, use them as privileged context, and update its model through verified student-generated attempts. This may avoid repeated probability-returning API calls and tokenizer-alignment work. It also avoids making the external model’s response style the direct training target.
The training-cost comparisons are informative, though incomplete. For Qwen3-4B over 150 updates, BSD requires 31.4 H200 GPU-hours versus 29.4 for vanilla OPD with the shared-tokenizer teacher. With the different-tokenizer teacher, BSD requires 48.7 GPU-hours versus 70.8 for xOPD. In the closed-API configuration, BSD uses 33.3 versus 83.0 for rubric-based OPD.
These figures compare specific implementations and exclude external teacher-trace generation, checkpoint saving, and evaluation. External API computation is not fully represented either. They are not end-to-end total-cost comparisons.
BSD also adds a self-teacher forward pass, reported at approximately 7–18 seconds per update and less than 10% of update time. Richer traces create additional generation, storage, and context-processing demands.
The data-design decision is therefore not simply whether to collect longer solutions. It is whether the additional information in those solutions improves the relevant outcome enough to justify its full cost.
Verifiable mathematics is the demonstrated boundary
The principal evidence comes from English competition mathematics, four Qwen3 student backbones no larger than 4B parameters, and automatically checked answers.
That boundary affects several deployment decisions.
First, a reliable verifier is central to BSD’s learning objective. The study does not establish comparable results for open-ended commercial tasks whose correctness cannot be determined automatically.
Second, the three teacher-access categories each use a different teacher. Teacher identity and access conditions are not independently varied, so differences cannot be attributed solely to API or tokenizer access.
Third, the principal benchmark table does not report uncertainty across independent training repetitions. Most component ablations are concentrated in one student–teacher configuration. Supplementary MMLU-Pro and legal-question evaluations provide some evidence of out-of-domain retention, but not comprehensive preservation of general capabilities.
For a development team evaluating BSD on its own tasks, the appropriate comparison would include SFT and applicable on-policy baselines, with separate measurements of single-answer accuracy, multi-attempt success, response length, out-of-domain behavior, and total training cost.
These are conditions for evaluating the method, not outcomes already demonstrated in production.
Learning from stronger models need not mean copying them
BSD’s main contribution is a way to use external reasoning without making the external model’s tokens or response patterns authoritative.
The teacher provides information. A copy of the student converts that information into compatible token-level guidance. The automatic verifier determines correctness, while guidance changes how unsuccessful attempts are penalized.
Across the tested mathematics configurations, this separation produces consistent average-accuracy improvements and works under substantially different external-teacher interfaces.
For compact-model developers with reliably checkable tasks, it expands the available post-training options beyond direct imitation and probability-dependent distillation. Whether it offers the best quality, behavior preservation, and total cost in a particular application remains an empirical decision.
Cognaptus: Automate the Present, Incubate the Future.
-
Runsheng Chen and Yihong Liu and Raoyuan Zhao and Michael A. Hedderich and Hinrich Schütze and Matthias Grabmair (2026). Bridged Self-Distillation for On-Policy Learning. arXiv:2610.26397. https://arxiv.org/abs/2610.26397 ↩︎