Cover image

The Teacher's Trace Is Not a Training Target

TL;DR for operators Access to a stronger AI model’s solutions does not guarantee that copying those solutions will improve a smaller model. In one experiment, supervised fine-tuning reduced a 0.6B-parameter student’s mathematical reasoning accuracy from 10.8% to 2.8%. An alternative training procedure raised it to 12.5%. Bridged Self-Distillation (BSD) uses external-model responses as privileged training context rather than outputs the student must imitate. A frozen copy of the student reads the external response and provides token-level guidance on the student’s own attempts. An automatic verifier still determines whether those attempts deserve reinforcement or penalties. ...

November 16, 2026 · 9 min · Zelina