The 70B Model May Belong Upstream

TL;DR for operators

If a team has a small labelled seed set and a large volume of multilingual text to classify, keeping the strongest LLM in every inference request may not be the best allocation of compute. Pecher et al. find that smaller models using examples generated by LLaMA-3 70B can exceed that same 70B model used directly as a zero-shot classifier with roughly 50 synthetic examples in aggregated language groups.¹

The result is strongest where the large model itself is relatively weak: low-resource languages, ten-class intent recognition, and other less represented tasks. With full synthetic pools, average gains over the generator baseline reach about 10 percentage points overall, and substantially more in some low-resource settings.

But synthetic data has a useful range rather than an unlimited scaling curve. At very small budgets, synthetic and human-labelled examples perform similarly. As more data becomes available, human-labelled training keeps improving while synthetic-data performance tends to stagnate. Synthetic training also produces more variance, failed runs, overfitting, and sensitivity to tuning choices.

For high-volume classification, the operational decision is therefore not simply “large model or small model.” It is where the large model belongs in the system: recurring inference, offline data generation, or an initial bootstrap stage before human-labelled data becomes the better marginal investment.

The expensive model does not have to be the endpoint

A familiar multilingual deployment problem starts with two constraints. The organization needs to classify large quantities of text, but labelled examples are scarce. A capable large LLM can classify each request without task-specific training, so leaving it in the inference loop appears to minimize development risk.

The experiments in Better as Generators Than Classifiers challenge that default role assignment.¹ LLaMA-3 70B is used twice: first as the zero-shot classifier to beat, and separately as a generator producing labelled examples for smaller models.

Those generated examples are not created from nothing. The generation process uses label descriptions and ten randomly selected human-labelled demonstrations, followed by duplicate removal and LLM-based self-revision filtering. The setup is therefore low-resource, not label-free.

The smaller models then consume the generated data in three different ways. XLM-RoBERTa Large is fine-tuned as a classifier. LLaMA-3.1-8B, Gemma-3-4B, and Qwen-2.5-7B receive synthetic examples through in-context learning or LoRA instruction-tuning. Experiments vary the amount of available data, balance samples across classes, and repeat conditions 20 times.

This is knowledge transfer through examples: the large model externalizes some of what it knows into labelled text, and a smaller model learns from that output. The paper refers to this as data-driven distillation.

Around 50 examples can change which model should classify

The central evidence is a break-even result: how much generated data does a smaller model need before it matches or surpasses the 70B model used directly?

Across aggregated language groups, smaller models can exceed the zero-shot baseline with as few as about 50 synthetic examples. The threshold is less clean when variance matters. For XLM-RoBERTa fine-tuning in high-resource languages, roughly 100–150 examples may be needed before performance consistently clears the large-model baseline.

With the complete synthetic pools, the average improvement is about 10 percentage points over the generator used directly as a classifier. The gains are uneven:

The relevant comparison is therefore not “Is a 70B model stronger than an 8B model?” It is whether the 70B model creates more downstream value by producing task-specific supervision than by answering every classification request itself.

For operators, that changes the architecture under consideration. A large model can become an offline teacher while a smaller model becomes the production classifier.

Task representation matters as much as language resource level

The gains are not uniformly distributed across tasks.

Synthetic data contributes little or no advantage for highly represented binary sentiment classification. Topic classification benefits more. Ten-class intent recognition produces the largest improvements: smaller models can outperform the generator with as few as ten examples, with reported gains reaching as high as 40 percentage points in low-resource languages and 30 points in high-resource languages.

That pattern matters because “low resource” is not purely a property of language. The paper’s additional English sarcasm experiment tests whether a niche task can exhibit similar behavior even in a high-resource language. Encoder fine-tuning on synthetic sarcasm data exceeds the generator by about 12 macro-F1 points, although results for compact-model in-context learning depend strongly on model family and sample selection.

This extension is useful as a boundary test rather than a second main thesis. It supports the interpretation that task representation also conditions whether generated supervision is valuable. It does not establish that pretraining representation is the causal mechanism.

For a company deciding where to try this architecture, language scarcity alone is therefore an incomplete screening criterion. Intent routing in an underserved language may be an excellent candidate. A familiar binary classification task already handled well by the large model may offer little return.

More synthetic data does not behave like more human data

The most consequential qualification appears when synthetic and human-labelled examples are compared under matched budgets and training procedures.

At very small budgets—roughly the first 50 to 100 examples—the two sources can yield similar improvements. Beyond that range, their trajectories separate. Human-labelled data continues improving, while performance from the unoptimized synthetic pools tends to flatten.

At the 400-example comparison, the synthetic-versus-human gap reaches as much as 10 percentage points in low- and medium-resource languages and about 5 points in high-resource languages. For in-context learning, the corresponding gaps are smaller, around 4 and 2 points.

The paper attributes the flattening to lower diversity and informativeness in generated samples, alongside greater overfitting and sensitivity to sample selection and hyperparameters. These explanations are plausible mechanisms supported by the experimental patterns, but the study does not directly isolate them causally and does not test a diversity-promoting generation strategy.

That distinction changes how synthetic data should enter an operating plan. It is well supported here as a bootstrapping resource. The evidence is much weaker for treating it as a permanent replacement for human annotation.

The cost case depends on which compact-model route you choose

Appendix C provides a separate computational comparison. Its purpose is operational rather than another performance test: it estimates how the architectural choices redistribute GPU time.

The encoder result makes the largest cost contrast. If its task performance is sufficient, shifting recurring classification from the 70B model to XLM-RoBERTa can radically change the compute profile.

Instruction-tuning requires a different calculation. It often produces strong performance, particularly in low-resource settings, but its reported 10-hour full-run cost approaches the generator’s 15 hours. Its advantage therefore depends on training being amortized across enough later inference.

Cognaptus would treat these numbers as inputs to a deployment-specific break-even calculation, not universal cost ratios. The study reports GPU wall-clock time on its experimental setup; it does not model API pricing, latency targets, hardware utilization, engineering labor, retraining frequency, or production request volume.

Production use requires a measured transition from synthetic to human supervision

The evidence supports a specific operating sequence.

Start with a small human-labelled seed set. Use a capable multilingual LLM to generate additional labelled examples. Train or condition a compact classifier. Measure where that classifier crosses the direct large-model baseline, including variance across repeated runs rather than only mean accuracy.

Then continue measuring the marginal value of new synthetic examples against new human-labelled examples.

That last comparison is essential because the paper shows that the better data source changes with scale. Synthetic supervision can reduce the amount of annotation required to reach a viable first model. It does not remove the case for human-labelled data once quality requirements rise.

Production validation should also include diversity audits, repeated training runs, hyperparameter checks, and failure-rate tracking. The study reports greater instability for synthetic-data training, including failed runs and below-random outcomes in some configurations. A mean improvement is insufficient if deployment performance depends heavily on sample draw or tuning choices.

The broader uncertainty is external validity. The study tests one 70B generator, one multilingual encoder, three compact LLMs, four tasks, 11 languages, and one common prompt design. That is substantial experimental coverage, but it does not establish a universal rule for LLM-generated supervision.

Allocate the large model to the stage where it creates the most leverage

The paper changes the deployment question from model size to model role.

When labelled data is scarce, a large multilingual LLM can be valuable precisely because it does not need to remain in the production path. Its outputs can become a temporary training asset that moves recurring classification onto a cheaper model. That strategy is most credible when the target language or task is poorly represented, classification volume is high, and a small synthetic dataset already produces a measurable break-even.

The transition should not end there. Once additional synthetic examples stop adding information and human-labelled data continues improving, the annotation budget becomes the more productive investment.

The resulting architecture is staged rather than static: use the large model to bootstrap supervision, deploy the smallest model that meets performance and stability requirements, and move toward human-labelled training when the synthetic-data curve begins to flatten.

Cognaptus: Automate the Present, Incubate the Future.