TL;DR for operators

A reasoning model can spend extra tokens reconsidering its answer and still fail to repair the mistake. Worse, unnecessary reconsideration can disturb reasoning that was already correct.

The paper studied here treats that as a training problem rather than an invitation to add another inference-time review loop. It first filters a model’s own critiques using verifiable ground truth, then trains the model on the surviving examples. A second reinforcement-learning stage rewards both final-answer correctness and the quality of the reflective step itself.

For teams building agents on tasks with reliable checkers, the operational lesson is conditional but concrete: teach the model what productive correction looks like before paying for more reflection at runtime. The reported evidence is strongest for mathematical and GPQA-style reasoning, weaker for 1.5B models, and does not establish transfer to open-ended decisions.

More checking is not the same as better correction

A familiar reasoning workflow looks sensible on paper: generate an answer, ask the model to reconsider it, then accept the revised result. The extra pass appears to buy caution.

The problem is that reflective language is only an observable behavior. It does not tell an operator whether the model found an actual error, introduced a new one, or simply spent more tokens narrating uncertainty.

That distinction becomes visible in Teaching Large Reasoning Models Effective Reflection by Hanbin Wang and colleagues.1 In a one-round critique evaluation, the untreated critic ended at 20.0% accuracy after critique. The paper’s fully trained variant reached 50.0%. Wrong-to-right transitions increased from 3.3% to 30.0%, while its correct-to-wrong transition rate was 3.3%.

The design question is therefore not how often a reasoning system should “think again.” It is how to make the second look reliably different from an untrained retry.

Filter self-critique before turning it into supervision

Raw self-critique has an obvious weakness: the same model that produced a bad solution can also produce a bad diagnosis of that solution.

The paper’s first stage, Self-Critique Fine-Tuning (SCFT), inserts a correctness gate before those critiques become training data. The model produces an answer and then critiques that answer. The critique is retained only when its resulting answer is correct against ground truth.

Two kinds of examples survive. An initially incorrect response may be corrected into a correct one, or an already-correct response may be validated without being damaged. The same model is then fine-tuned to generate those retained critiques conditioned on the question and its original response. Ordinary question-answer pairs are not added as the supervision target in this stage.

That filtering step is doing much of the conceptual work. The method does not assume that self-generated reasoning traces are trustworthy merely because they sound diagnostic.

The data-composition ablation reinforces the point. With 1,000 error-correction examples held fixed, the best reported 7B result uses 500 correct-answer validation examples—a 2:1 balance. Average Pass@1 reaches 61.7 and ERR 0.43. Increasing the validation set further does not improve performance monotonically.

This is an ablation, not a second headline result. Its purpose is to test whether critique supervision behaves like a volume problem. Within this setup, it does not: composition matters.

An auxiliary critique-source comparison adds another boundary. Critiques from the stronger DeepSeek-R1 model can improve some critique-fine-tuning results beyond self-generated critiques. But the core SCFT mechanism does not require that stronger teacher. The paper’s contribution is the filtered self-critique route, not evidence that teacher quality has become irrelevant.

Reflection needs its own reward signal

Once a model has learned examples of productive correction, final-answer reward alone might seem sufficient for reinforcement learning. The paper tests that assumption directly.

Its second stage, RLERR, initializes reinforcement learning from the SCFT-trained policy and uses GRPO. A strong LLM judge evaluates reflective behavior according to five ordered properties: truthfulness, constructiveness, specificity, substantiveness, and information gain. That process-oriented reward is combined with an outcome reward for answer correctness.

The reward ablation is particularly informative. Reflection-only and outcome-only variants stagnate around 42–44% on the reported AIME2024 experiment, while the combined method peaks near 49%. The likely purpose of this experiment is narrow: determine whether the two reward channels are redundant. They are not in the tested setup.

A separate initialization ablation reports that starting RL from the SCFT model reaches a higher Pass@1 ceiling than starting from the base or self-distilled model. Taken together, the two tests support a staged interpretation: first establish a usable correction policy, then optimize when and how that policy is expressed.

That is different from relying on reinforcement learning to discover the entire behavior from final outcomes.

Measure whether reflection accompanies success, not how much of it appears

The paper also introduces Effective Reflection Ratio (ERR):

$$ \text{ERR}= \frac{\sum_{i=1}^{n}\text{Accuracy}(y_i)\cdot R(y_i)} {\sum_{i=1}^{n}R(y_i)} $$

In plain language, ERR measures the fraction of detected reflection occurrences that appear inside fully correct responses.

This makes it more informative than simply counting reflective phrases or reasoning steps. For the 7B DeepSeek-R1-Distill-Qwen model, average ERR rises from 0.38 at baseline to 0.43 after SCFT and 0.54 after RLERR. Average Pass@1 moves from 58.5 to 61.7 and then 65.1.

The metric still has an interpretation boundary. ERR does not prove that each reflection caused the correct answer. It assigns reflection inside a correct trajectory to the successful side of the aggregate ratio. For production monitoring, that makes ERR closer to a behavioral association metric than a causal measure of correction value.

The ablations identify which parts of the recipe carry weight

Test Likely purpose What it supports What it does not prove
Error-correction vs. validation mix Ablation Critique composition matters; 2:1 performs best in the reported 7B setup That 2:1 is universally optimal
Self vs. stronger-model critiques Comparison / extension Better critique sources can further improve training That SCFT requires an external teacher
Base, self-distilled, and SCFT RL starts Initialization ablation SCFT provides a stronger starting policy in the reported 1.5B RL experiment That the advantage holds for every backbone
Outcome vs. reflection vs. combined rewards Reward ablation Process and outcome signals are complementary in the tested RL setting That the same judge or reward hierarchy generalizes to open-ended work

The overall benchmark table is consistent with that story, but scale matters. SCFT moves the 14B model from 68.1 to 69.8 average Pass@1 and from 0.51 to 0.57 ERR. The gains are much smaller for the tested 1.5B models. RLERR is also evaluated only on DeepScaleR-1.5B-Preview and the 7B DeepSeek distill model because of compute limits.

The evidence therefore supports a training design under specific model and task conditions, not a scale-independent law of reflection.

For verifiable agents, train the correction loop before buying more runtime tokens

Cognaptus inference: teams operating reasoning systems with deterministic or inexpensive verification can treat self-correction as part of the training architecture rather than only as an inference policy.

A plausible implementation sequence follows directly from the paper’s mechanism:

  1. Let the policy generate both solutions and critiques on tasks with reliable answer checks.
  2. Reject critiques that fail the checker rather than treating all reflective traces as supervision.
  3. Fine-tune on the retained correction and validation trajectories.
  4. Use that policy as the starting point for RL.
  5. Reward both task success and properties of the correction process.
  6. Monitor whether reflection is associated with successful trajectories, not merely whether reflection frequency increases.

For a team deciding between another test-time review pass and additional training work, the relevant condition is verifier availability. If answers can be checked cheaply and consistently, the paper provides evidence for moving some investment upstream into learned correction behavior.

Where correctness is subjective, delayed, multi-objective, or dependent on human judgment, the rejection gate becomes much harder to reproduce. An ERR-like production metric also becomes less interpretable when “correct” cannot be represented reliably as a binary outcome.

The boundary is verifiability, not just model quality

Three constraints materially affect deployment interpretation.

First, ground-truth rejection sampling assumes that both the original answer and the critique output can be checked. That condition fits mathematics and some structured coding, planning, or tool-use tasks better than strategy, writing, negotiation, or other open-ended decisions.

Second, SCFT produces limited improvements in the tested 1.5B models. The paper interprets this as possible capacity limits in exploiting self-critique signals, so smaller-model deployments should not assume the same return from the supervised stage.

Third, the benchmark comparisons do not report significance tests or uncertainty intervals. The numerical pattern is supported by multiple benchmarks and several targeted ablations, but small differences should not be read with more precision than the evaluation supports.

The broader lesson survives those boundaries. Reflection has a cost, and its presence is not evidence of its value. Where outcomes are verifiable, this paper offers a stronger design target: train the system to recognize and execute productive correction, then reward that behavior alongside the answer it eventually reaches.

Cognaptus: Automate the Present, Incubate the Future.

  1. Hanbin Wang and Jingwei Song and Jinpeng Li and Qi Zhu and Fei Mi and Ganqu Cui and Yasheng Wang and Lifeng Shang (2026). Teaching Large Reasoning Models Effective Reflection. arXiv:2601.12720. https://arxiv.org/abs/2601.12720 ↩︎