The Data Diet for Reasoning Models: Why Less (But Smarter) Wins

A model-training team has a familiar bad habit: when the model fails, it asks for more.

More examples. More domains. More synthetic prompts. More compute. More benchmarks to average over until the unpleasant details become small enough to ignore.

This habit is understandable. It is also expensive. And, according to SuperNova, it may be the wrong first instinct.

The paper, SuperNova: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructions, studies how to train LLMs for general reasoning using reinforcement learning with verifiable rewards, or RLVR.¹ Its central move is not a new model architecture, a mysterious alignment ritual, or another heroic scrape of the internet. It asks a more practical question: can existing human-annotated instruction-tuning data be converted into verifiable RL training data, then selected and mixed carefully enough to improve reasoning?

The answer is yes, but with a catch that should make many data pipelines slightly nervous: not all “reasoning data” is equally useful. Some tasks help. Some tasks barely matter. Some tasks can degrade the model. And broad categories such as “logic,” “math,” “commonsense,” or “multi-hop” are too crude to decide what deserves a place in the training mixture.

This is why SuperNova is more interesting as a comparison paper than as a framework announcement. Its value comes from the contrasts it sets up: STEM reasoning versus general reasoning, macro mixing versus micro mixing, original curated data versus synthetic interventions, and smaller models with better data versus larger models with weaker data. The paper’s quiet message is not “data quality matters.” We already knew that, at least ceremonially. The sharper message is that reasoning data has to be treated like a portfolio, not a warehouse.

The easy mistake: confusing domain diversity with reasoning coverage

RLVR became attractive because it works cleanly when answers can be checked. Math and code are the natural playgrounds. A final answer can be compared, a unit test can be run, a reward can be assigned, and the model can be trained to search toward better solutions.

General reasoning is messier. A business analyst may need causal inference, temporal ordering, pragmatic interpretation, contradiction detection, spatial reasoning, and constraint tracking in the same afternoon. The answer may still be verifiable, but the training data is harder to source and harder to filter. Web-scraped domain QA gives you more topics, but more topics do not automatically mean more reasoning skills. “Finance,” “history,” and “science” are domains. They are not skills.

SuperNova’s starting observation is useful because it looks backward before inventing forward. Instruction-tuning datasets such as SuperNI already contain many expert-annotated tasks. These tasks were originally built for instruction following, not RLVR. But many contain structured reasoning patterns: pronoun disambiguation, causal classification, temporal commonsense, textual entailment, analogical reasoning, and multi-hop question answering.

The obstacle is that instruction-tuning data is not automatically usable for RLVR. Open-ended answers may be hard to verify. Some examples are too easy to create useful learning signal. Others are too hard for the model to learn from under the training setup. Some tasks simply do not transfer to the target reasoning benchmark.

SuperNova turns this into a pipeline:

1. Select candidate tasks from SuperNI.
2. Reformat task examples into verifiable question-answer pairs.
3. Filter examples whose win rate is either 0 or 1 under the baseline model, because they are respectively too hard or too easy.
4. Train task-specific models to estimate downstream task utility.
5. Mix the best tasks either globally or per target sub-skill.
6. Test whether synthetic interventions improve the selected data.
7. Train final SuperNova models and evaluate them on BBEH and unseen reasoning benchmarks.

The workflow sounds obvious only after someone has done the expensive part. As usual, the obvious thing becomes obvious right after the controlled experiment makes it inconvenient to ignore.

Comparison 1: STEM reasoning does not automatically transfer to general reasoning

The first comparison is against the industry’s favorite shortcut: train on math and code, then hope the model becomes generally thoughtful. That hope has some intuitive appeal. Math training rewards multi-step problem solving. Code training rewards precision. Surely those habits should transfer.

SuperNova is less impressed.

The paper argues that strong performance in formal domains does not reliably transfer to general reasoning tasks such as causal inference, temporal understanding, pragmatic interpretation, and commonsense reasoning. The authors position this against prior reasoning models trained heavily on STEM-style data, noting that such models can improve on math benchmarks while underperforming on general reasoning benchmarks such as BBEH.

For business readers, this distinction matters. Many enterprise workflows are not math contests in disguise. A customer-support escalation, compliance memo, procurement exception, or policy interpretation task may involve reasoning, but the reasoning often comes from tracking conditions, reconciling context, understanding exceptions, and avoiding plausible but wrong interpretations.

A model that improves on competition math may still fail at “what follows from this messy policy paragraph?” That is not because math is useless. It is because the training signal may not touch the right capability.

This is the first practical correction: the unit of design should not be “domain.” It should be “reasoning capability under a verifiable task format.”

Comparison 2: task selection beats task accumulation

SuperNova’s most important experiment is also the least glamorous. The authors train Qwen3-0.6B on individual candidate tasks under compute-matched settings and measure the effect on BBEH-mini. The result is not a gentle spread. Task choice materially changes downstream performance.

The paper reports a 7.6 percentage point gap between the lowest-performing task, task213-rocstories, and the highest-performing task, task738-perspectrum, in the task-selection experiment. Several tasks degrade performance relative to the baseline.

That last sentence is the expensive one.

The usual enterprise instinct is to collect all available labeled tasks, clean obvious junk, and assume the rest will average out. SuperNova says no. Averaging can bury the signal. Worse, weak tasks do not merely sit there politely doing nothing. Under RL, they can redirect optimization toward habits that do not help the target reasoning capability.

The appendix adds a useful nuance. Coarse category labels are not sufficient. Multi-hop reasoning and coreference resolution emerge as strong categories, while narrative and surface-formatting tasks tend to underperform. But even this category-level view hides variation. Textual entailment and NLI may rank only in the middle as a broad group, while task738_perspectrum becomes the top-ranked task by a large margin.

This is a classic data governance problem wearing an AI lab coat. The label on the folder is not enough. You have to measure what the examples actually do.

The business implication is straightforward, though not always welcome: data selection needs evaluation loops. A dataset is not an asset simply because it is labeled. It becomes an asset when its marginal contribution is measured against the target capability.

Comparison 3: micro mixing beats macro averaging

Once useful tasks are identified, the next question is how to combine them. SuperNova compares two strategies.

Macro mixing ranks source tasks by average performance across the validation benchmark and selects the globally top tasks. Micro mixing ranks source tasks separately for each BBEH sub-task, then selects the top tasks per sub-task and combines the unique selected tasks.

Macro mixing is administratively attractive. One leaderboard, one ranking, one clean answer. Unfortunately, reasoning skills are not an HR performance review. A task that is excellent for causal understanding may not be the best for object tracking. A task that helps temporal reasoning may do little for disambiguation.

The reported results make this concrete:

The best result is Micro-Top2 at 22.8 pass@8. The margin over Macro-Top2 is modest, 1.1 points, but the pattern matters: selecting tasks per sub-skill better preserves capability coverage than selecting only by average score.

There is also a second lesson hidden in the table. More mixing is not automatically better. In both strategies, performance declines when too many tasks are added. The curve is not “more tasks, better reasoning.” It is closer to “enough diversity, then dilution.”

For enterprise adaptation, this is the part worth stealing.

A company does not need one giant “reasoning dataset.” It needs a capability map. Contract review may require clause retrieval, exception handling, temporal dependency, numerical threshold checking, and risk classification. Customer operations may require intent disambiguation, policy matching, causal diagnosis, and escalation judgment. Training and evaluation should preserve these sub-skills rather than compress them into one average score.

Macro averages make managers comfortable. Micro mixtures make models better. Cruel, but efficient.

Comparison 4: synthetic difficulty is not automatically better data

The paper then tests a fashionable idea: if curated data is good, perhaps making it synthetically harder will make it better.

The interventions are sensible. The authors try transformations such as adding many-hop reasoning, going against a strong prior, learning a new rule on the fly, padding long context, inserting flawed reasoning traces, inducing patterns from examples, adding constraint satisfaction, compositional understanding, and knowledge-intensive context. Importantly, these interventions modify the problem statement while preserving the target answer, because RLVR still needs verifiability.

The result is inconvenient: none of the interventions outperform the original Micro-Top2 dataset under the fixed training budget.

This does not mean synthetic augmentation is useless. That would be too convenient a conclusion, and therefore suspicious. The narrower reading is better: once a dataset is already selected for task utility and verifiability, naive synthetic complexity can damage the learning signal under a fixed budget.

Why might that happen? The paper does not prove a single mechanism, but the result is plausible. If the original task already contains the useful reasoning pattern, adding artificial difficulty may introduce noise, distractors, or distribution shift. The model then spends optimization budget learning around the intervention rather than strengthening the target capability.

For business teams, this is a warning against “prompt decoration as data strategy.” Making examples longer, trickier, or more elaborate is not the same as making them more useful. Sometimes synthetic sophistication is just noise wearing a lab coat.

Comparison 5: smaller curated models can beat larger baselines

The headline performance comparison is where SuperNova becomes easiest to explain to executives, though it should be interpreted carefully.

On BBEH-test, SuperNova-trained models improve over their Qwen3 baselines across model sizes. The 4B SuperNova model reaches 17.6 pass@1 and 33.3 pass@8. By comparison, Qwen3-4B reaches 13.6 pass@1 and 23.2 pass@8, while Qwen3-8B reaches 14.1 pass@1 and 24.2 pass@8. General-Reasoner-4B reaches 12.0 pass@1 and 32.9 pass@8.

The obvious line is that SuperNova-4B beats Qwen3-8B. That line is true, but the better interpretation is not “small models are better.” The better interpretation is “a smaller model trained on better-selected reasoning data can outperform a larger model that has not received the same capability-targeted signal.”

This distinction matters. Model size still helps in many settings. SuperNova does not repeal scaling laws; it reminds us that scaling the wrong input is a very elegant way to waste money.

The paper also evaluates out-of-distribution benchmarks that were unseen during data curation: MMLU-Pro, BBH, Zebralogic, and MATH500. SuperNova models improve average pass@8 across all model sizes. The 4B model rises from a Qwen3-4B average of 71.3 to a SuperNova-4B average of 80.1. The Zebralogic gain is especially large: 55.6 for Qwen3-4B versus 77.0 for SuperNova-4B.

This table is important because it weakens the worry that SuperNova merely overfits BBEH-mini. It does not eliminate the worry entirely, but it gives the result more credibility. The paper’s appendix also shows that gains are uneven across sub-tasks: seven of the 23 BBEH sub-tasks show negligible gains, while SuperNova improves areas such as Hyperbaton, Multi-step Arithmetic, and Shuffling Objects where the base model has near-zero performance.

That unevenness is not a flaw to hide. It is the point. Reasoning is not one dial.

The paper’s evidence map: what each test is really doing

A useful way to read this paper is to separate the role of each experiment. Otherwise, every table becomes “evidence,” and the reader loses the difference between the main argument and the supporting checks.

This evidence map keeps the argument disciplined. The paper does not prove that SuperNova is the final recipe for all business reasoning. It does show that careful task selection and capability-aware mixing produce stronger results than several intuitive alternatives.

What this means for enterprise AI teams

The enterprise lesson is not “use SuperNova.” Most companies will not reproduce this exact pipeline with BBEH-mini, Qwen3 models, and SuperNI-derived tasks. The lesson is operational: reasoning improvement should be managed as a data-portfolio problem.

The first step is to decompose the target workflow. “Improve legal reasoning” is too vague. The useful version is: identify obligations, compare exceptions, track dates, distinguish party roles, detect contradictions, and classify escalation risk. Each sub-skill needs examples, evaluation, and a way to measure marginal contribution.

The second step is to convert existing human-reviewed work into verifiable tasks where possible. Many companies already own examples of resolved tickets, reviewed claims, annotated compliance decisions, corrected analyst outputs, and labeled workflow outcomes. These are not automatically RLVR-ready, but SuperNova’s framing suggests a path: restructure them so the final answer can be checked, then filter out examples that are trivial or impossible for the model.

The third step is to run small utility tests before scaling. This is culturally difficult because it slows down the satisfying ritual of building a giant dataset. But it is cheaper to discover that a task is useless before it becomes 12% of your training mixture.

A practical enterprise version of SuperNova would look like this:

The ROI story is also more precise than “smaller model cheaper.” The real savings come from cheaper diagnosis. If a model fails after being trained on a giant undifferentiated dataset, nobody knows whether the problem is task mismatch, noisy labels, insufficient compute, weak base model capacity, or evaluation leakage. A portfolio approach creates levers. You can ask which sub-skill failed, which task mixture supported it, and whether adding more of a task helps or hurts.

That is not glamorous. It is merely how serious systems are built.

Boundaries: what SuperNova does not prove

The paper’s limitations are not decorative; they affect how the result should be used.

First, the main evaluation is benchmark-based. BBEH, BBH, Zebralogic, MMLU-Pro, and MATH500 are useful stress tests, but they are not production workflows. A model that improves on constraint puzzles may still fail when the input is a messy email thread with missing attachments and office politics quietly leaking through the wording.

Second, much of the curation study uses Qwen3-0.6B for tractability, then scales the final recipe to larger Qwen3-family models. The paper includes cross-model evidence on LLaMA3.2-3B-Instruct and Qwen3.5-2B, which is encouraging, but not enough to claim universal transfer across architectures, model sizes, or deployment settings.

Third, the compute budget is fixed and finite. The finding that synthetic interventions underperform the original curated data holds under this setup. It does not prove that all synthetic data interventions fail under all budgets, prompts, filtering methods, or training regimes.

Fourth, the pipeline itself uses frontier-model assistance for candidate selection and reformatting. The authors manually inspect 100 reformatted samples and report perfect adherence in that sample, but a production version would need stronger quality controls, especially when outputs affect regulated decisions.

Finally, SuperNova improves reasoning scores; it does not directly evaluate safety, calibration, refusal behavior, compliance reliability, or business KPI lift. Those are separate tests. Annoying, yes. Optional, no.

The better mental model: data selection as capital allocation

The most useful business metaphor here is not “data diet,” although the title is too good to abandon. It is capital allocation.

A training budget is finite. Every example consumes optimization attention. Every task included in the mixture competes with every other task. A broad dataset can look diversified while actually overweighting weak signals. A smaller curated mixture can outperform it because the examples are doing different jobs for different capabilities.

That is the paper’s deeper contribution. SuperNova does not merely say “use better data.” It shows that better data requires measurement at the task level, mixing at the sub-skill level, and skepticism toward synthetic difficulty that has not earned its place.

For AI builders, the uncomfortable implication is that “we have a lot of data” is no longer a serious sentence. The better question is: which capabilities does the data train, how do we know, and what happens when we remove it?

Less can win. But only if “less” is selected with evidence rather than vibes.

Cognaptus: Automate the Present, Incubate the Future.