Grow While You Roll: Training the Draft Head Inside the RL Run
TL;DR for operators LLM reinforcement learning spends a large share of its wall-clock budget generating responses. Speculative decoding can reduce that cost, but it normally assumes that a smaller predictor is already capable of proposing useful tokens before the main model verifies them. GrowMTP1 tests whether that capability can instead be learned during the RL workload itself. On Qwen3-4B, a randomly initialized five-token draft head reached an acceptance length of 2.91 after 500 mathematical-reasoning RL steps. Rollout generation became 2.13x faster, and the complete RL step—including draft-head training—became 1.60x faster. ...