Traces of War: Surviving the LLM Arms Race

TL;DR for operators

Reasoning traces are useful. That is the problem.

When a frontier reasoning model shows its work, it gives customers more confidence, gives developers more debuggability, and gives downstream applications a richer interface than a bare answer. It also gives competitors and opportunistic scrapers a training asset. The trace is not just an explanation; it is labelled behavioural data from an expensive model. Very polite leakage, in other words.

The paper behind this article, Antidistillation Sampling, proposes a decoding-time defence: keep producing reasoning traces that still work for the teacher model, but perturb token sampling so those traces become poor training data for a student model trying to distil the teacher.¹ This is not “hide the chain-of-thought” and it is not “increase randomness until the output becomes useless”. The mechanism is more pointed. The teacher uses a hidden proxy student model to estimate which next tokens would damage downstream student learning, then adds that signal into the sampling distribution.

Operationally, the paper shows three things worth taking seriously. First, antidistillation sampling gives model owners a tunable trade-off between teacher utility and distillation resistance. Second, on GSM8K, MATH, and MMLU, it reduces distilled-student accuracy more effectively than temperature sampling at comparable teacher-performance loss. Third, the method still works when the proxy and actual student are not the same model family, which matters because real attackers do not RSVP with their architecture choices.

Cognaptus’ read: this is an early design pattern for “defensive inference”. The strategic unit is no longer only the model weights, the training data, or the API rate limit. It is also the decoding policy. That is where product value and IP leakage meet, awkwardly, every time a model speaks.

The boundary is equally clear. The experiments are proof-of-concept, not a production security guarantee. The setup uses benchmark reasoning tasks, relatively modest model sizes, and a proxy-model procedure that adds compute. It is promising because the mechanism is specific. It is limited for the same reason.

The trace is both product feature and training material

The obvious business answer to model stealing is to reveal less. Do not expose logits. Do not expose system prompts. Do not expose full reasoning. Put everything behind a black-box API, add rate limits, and hope the moat holds.

That answer has a cost. Reasoning traces are not decorative. In high-stakes or workflow-heavy settings, they make model behaviour more inspectable, easier to route, easier to debug, and easier to trust. A bare answer may be enough for trivia. It is not enough when an enterprise user wants to understand why an agent rejected an invoice, proposed a trading rule, classified a contract clause, or selected a maintenance action. The “show your work” interface became valuable precisely because the work matters.

The paper’s core tension sits there: the same traces that make a reasoning model commercially usable also make it easier to distil. A student model trained on a teacher’s generated traces can absorb useful behaviour without paying the full original training cost. The teacher’s expensive reasoning process becomes someone else’s cheap supervised fine-tuning dataset. Capitalism, but with nicer GPUs.

The authors frame antidistillation sampling as a way to avoid a bad binary choice. Instead of either exposing clean traces or hiding traces entirely, modify the sampling process so the trace remains plausible and useful for the teacher’s task while becoming less helpful to a future distiller.

That distinction matters. The paper is not mainly about making outputs bad. Anyone can do that. The interesting question is whether a model can produce outputs that are still good for the user but bad for unauthorised model training.

The mechanism: perturb the next token, not the whole product

Antidistillation sampling lives inside decoding. At each generation step, the teacher model already has a next-token distribution. Standard decoding samples from that distribution, often with temperature scaling. Antidistillation adds another term before sampling.

The added term is not random. It comes from a proxy student model.

The teacher owner does not know what architecture an attacker will use. So the method assumes a proxy: a smaller or separate model maintained by the defender, used to estimate how a student might learn from generated traces. The proxy is evaluated on a downstream loss, such as negative log-likelihood over held-out reasoning traces. The defender then asks a surgical question:

For each possible next token, would training on this token push the proxy student in a direction that improves or worsens its downstream performance?

Tokens that are likely to worsen downstream student performance receive a positive antidistillation adjustment. Tokens that look too helpful for distillation are less favoured. The teacher still favours tokens that are probable under its original distribution, so the generation does not simply collapse into nonsense.

A simplified version of the sampling objective is:

Here, $\theta_T$ is the teacher model, $\tau$ is temperature, $\lambda$ controls the strength of the antidistillation penalty, and $\hat{\Delta}$ is the estimated antidistillation signal from the proxy model. Larger $\lambda$ means stronger protection but more risk to teacher utility. As ever, there is no free lunch. There is only a better priced lunch.

The expensive version of the idea would require computing the downstream-loss impact for every candidate token directly. That is not practical. The paper’s implementation uses a finite-difference approximation. It first computes the gradient of the proxy’s downstream loss, then compares the proxy’s next-token log probabilities under two perturbed copies of the proxy parameters:

This is the paper’s key engineering move. It converts a theoretically expensive “what would this token do to downstream student learning?” calculation into two proxy forward passes during generation, after a one-time downstream-loss gradient calculation.

That is why the method is a decoding-time defence rather than a retraining recipe. It does not require changing the teacher’s weights. It changes how the teacher speaks.

The misconception: this is not just noisier chain-of-thought

A quick but wrong summary would be: “The paper poisons chain-of-thought by making it noisy.”

That misses the mechanism.

Raising temperature also makes outputs noisier. Random perturbations also make logits less predictable. But neither knows which perturbations are harmful to distillation. They degrade the surface distribution, not the student’s learning trajectory. Antidistillation sampling is aimed at the latter.

The paper’s permutation baseline is useful here. The authors construct a noisy baseline that preserves statistical properties of the antidistillation perturbations but scrambles the information by random permutation and sign flipping. This asks: is the defence working because the logits are merely being shaken, or because the gradient-informed direction matters?

The answer is that the direction matters. Permutation sampling can hurt distillation more than ordinary temperature in some regions, but it does not match the antidistillation curve. Destroying the gradient information weakens the effect. That makes the appendix more than housekeeping. It is an ablation of the central claim: the defence is not “noise”; it is targeted misinstruction.

A useful mental model:

The main evidence: the student suffers before the teacher collapses

The paper’s headline comparison is between antidistillation sampling and temperature sampling. Temperature is a fair baseline because it is also a decoding-time intervention and can be tuned to reduce teacher performance. The question is whether the same teacher-utility sacrifice buys more distillation resistance.

On the paper’s illustrative MMLU example, a teacher at 72% accuracy yields a naively distilled student reaching up to 52%. Increasing temperature can reduce teacher accuracy by around 4 percentage points while leaving the distilled student roughly unchanged at 52%. Antidistillation sampling with a similar teacher-accuracy reduction lowers student accuracy to as low as 40%.

The GSM8K schematic in Figure 1 makes the trade-off even more concrete. Under standard sampling, the teacher is shown at 91% accuracy, the base student at 31%, and the distilled student at 61%. Under antidistillation sampling, the teacher is shown at 70%, the same base student remains at 31%, and the poisoned student falls to 25%. That is a larger teacher sacrifice than an operator would usually tolerate in production, but it clarifies the mechanism: the student is not merely failing to improve; it can be pushed below its undistilled baseline.

The more practical regime is the high-teacher-accuracy zoom. The authors explicitly examine the first 5% drop in teacher accuracy because model providers will not happily burn core product quality for a theoretical moat. In that zoomed region, the paper reports that moving GSM8K teacher accuracy from 90% to 89% drops poisoned-student accuracy from 65% to 56%, while temperature sampling does not degrade the student at all.

That is the business-relevant shape of the result. Not “we can ruin the output and ruin the student”. More interestingly: a small controlled loss in teacher utility may create a disproportionate loss in distillation value.

The experiments are a chain of purpose, not a pile of plots

The paper’s empirical section is best read as a sequence of checks around one mechanism.

This is a strong structure for an early paper. The authors do not merely show one curve and declare victory. They ask whether the effect depends on architecture matching, whether the approximation is plausible, whether the perturbation is just noise, and whether the student is actually learning something pathological rather than simply failing to train.

The holdout-loss curve is especially important. The student’s training loss decreases while holdout loss increases. That means the traces are learnable, but what is learned does not transfer into better downstream reasoning. For distillation defence, that is exactly the unpleasant sweet spot: the attacker’s training process appears to proceed, but the resulting capability is worse.

The proxy model is the clever part—and the fragile part

The method relies on a proxy model because the defender does not know the attacker’s student architecture. This is realistic. A model provider cannot assume the attacker is using Llama, Qwen, a proprietary internal model, a mixture, or tomorrow morning’s freshly released “definitely open” miracle.

The paper’s primary setup uses DeepSeek-R1-Distill-Qwen-7B as the teacher, Qwen2.5-3B as the proxy, and Llama-3.2-3B as the student. That proxy/student mismatch is not incidental. It is a test of whether proxy-directed poisoning transfers across model families. The results suggest it can.

The appendix extends this by trying all-Qwen and all-Llama configurations on GSM8K, plus smaller and larger Qwen proxy models against a Llama student. The results remain broadly similar. That supports a practical claim: the defender may not need to know the exact student architecture to degrade distillation.

But this is also where deployment risk lives. Proxy choice becomes a security parameter. A proxy that is too small, too misaligned with likely attackers, or trained/evaluated on the wrong downstream loss may produce weak antidistillation signals. A proxy that is too large raises inference cost. A proxy objective that protects only benchmark-style math reasoning may not protect code generation, tool use, legal analysis, or domain-specific planning.

The model owner must therefore answer a product-specific question: what capability are we trying to make harder to steal?

“Reasoning” is not a single asset. It may mean arithmetic, symbolic manipulation, coding style, financial analysis, agentic tool planning, rubric following, or safety deliberation. Antidistillation sampling needs a downstream loss that represents the capability worth protecting. Choose that badly and the mechanism may diligently defend the wrong castle. Very medieval, but in Python.

The compute cost is visible, not fatal

The paper is refreshingly direct about overhead. Antidistillation sampling requires two forward passes through the proxy model for each teacher forward pass, independent of $\lambda$. In the main experiments, the proxy model is approximately half the teacher’s size, so the authors describe the overhead as roughly doubling the computation needed to sample outputs compared with temperature sampling.

That is not a footnote for operators. It is the invoice.

Still, the cost should be interpreted carefully. First, this is an inference-time control, not a full retraining cost. Second, the proxy can potentially be smaller than the teacher, and the paper’s proxy-size experiments suggest smaller proxies can still be effective in tested settings. Third, not every request needs the same protection level. A model provider might apply stronger antidistillation only to high-risk endpoints: long reasoning traces, code explanations, specialised analytic workflows, or accounts exhibiting scraping-like behaviour.

This suggests a product architecture rather than a universal decoding setting:

The point is not that every token should be defended equally. The point is that decoding becomes a policy layer. That is where commercial judgement enters.

What this directly shows, and what business must infer

The paper directly shows that antidistillation sampling can degrade student distillation performance on GSM8K, MATH, and MMLU under the tested model configurations. It also shows that the trade-off can be tuned through $\lambda$, that finite-difference approximation is workable in the reported setup, that proxy-to-student generalisation can occur across model families, and that randomised perturbation baselines do not explain away the result.

Cognaptus infers three business implications.

First, reasoning-output policy becomes IP policy. A frontier model provider should not treat trace exposure as a purely UX decision. The richer the trace, the richer the potential distillation dataset. Antidistillation sampling gives providers a way to think beyond “show everything” versus “show nothing”.

Second, API defence is moving closer to the generation process itself. Rate limits, account monitoring, legal terms, and watermarking all operate around the model. Antidistillation changes the model’s emitted training signal. That is a deeper intervention, and likely a more uncomfortable one for attackers because it targets learning efficiency rather than access alone.

Third, protection can be task-specific. The downstream loss used by the proxy model can, in principle, encode the protected capability. For business use, that means a provider might defend code reasoning differently from financial reasoning, mathematical reasoning, or safety reasoning. One moat, many locks. Annoying for engineering, useful for strategy.

What remains uncertain is just as important. The paper does not prove robustness against adaptive distillers who know or suspect antidistillation is being used. It does not test frontier-scale closed models. It does not establish that protected traces remain equally valuable for human interpretability. It does not solve the governance problem of whether poisoning outputs—however strategically justified—could create unexpected downstream harms when those outputs enter user workflows, logs, datasets, or audits.

The practical message is therefore not “deploy this tomorrow everywhere”. It is “start treating decoding as part of model security design”.

The appendix is not a second thesis; it is a stress test of the mechanism

A common mistake when reading ML papers is to treat appendices as miscellaneous storage. Here, the appendix is doing useful argumentative work.

The permutation baseline asks whether antidistillation is just noise. It is not.

The hyperparameter-$\epsilon$ check asks whether the finite-difference approximation behaves sensibly. It does, within a model- and precision-dependent sweet spot. The authors note that too small an $\epsilon$ creates round-off error, while too large an $\epsilon$ creates truncation error. That is an implementation boundary, not academic ornamentation.

The JVP comparison asks whether the formal gradient method could replace the finite-difference approximation. In their implementation, JVPs face practical support and memory issues, including interaction with attention kernels and precision. The finite-difference method is not merely a mathematical convenience; it is a deployment-friendly approximation.

The architecture and proxy-size appendices ask whether the method is brittle. The answer is “not obviously”, which is not the same as “solved”. Broadly similar results across Qwen, Llama, smaller proxy, and larger proxy setups support the mechanism’s generality, while still leaving plenty for production validation.

The example traces are also revealing. Some antidistillation traces remain perfectly useful; others contain odd fragments before recovering the correct answer. This is the human-facing cost hiding behind aggregate accuracy. A benchmark may score the final boxed answer as correct, while a user may notice strange intermediate text. Product teams should not ignore that distinction. Users buy the whole interaction, not only the final token inside \boxed{}.

For model providers, the control surface is now three-dimensional

Most AI security conversations around model access focus on whether the user can query the model and how often. Antidistillation sampling adds another axis: how distillable the response should be.

A provider could, in principle, tune three variables:

1. Disclosure: how much reasoning or intermediate work to show.
2. Fidelity: how close the output should stay to nominal teacher behaviour.
3. Distillability: how useful the output should be as training data for a student.

The traditional approach collapses the third variable into the first. Less disclosure means less distillation risk. Antidistillation sampling separates them partially. It says: perhaps we can keep disclosure while reducing distillability.

That partial separation is the strategic contribution. Not perfect secrecy. Not magical anti-copying. Just a new knob, placed exactly where the commercial tension lives.

For enterprise AI vendors, this could become part of API tiering. A research customer may want clean reasoning traces and accept contractual restrictions. A consumer product may prefer concise answers with lower trace exposure. A high-value reasoning API may use dynamic antidistillation by default. Regulated customers may demand auditable behaviour and reject any trace perturbation that affects interpretability. There is no universal answer because “security” and “usefulness” are not the same product metric.

Boundaries before procurement gets excited

There are four boundaries worth keeping in view.

First, the evidence is benchmark-centred. GSM8K, MATH, and MMLU are useful reasoning benchmarks, but they are not the full surface area of commercial LLM use. Real workflows involve tools, retrieval, multi-turn state, domain-specific constraints, and messy user prompts. Antidistillation may transfer, but transfer is a result to be earned, not assumed.

Second, the model scale is modest relative to frontier production systems. The paper uses open-weight models around the 3B–7B range in its experiments. That is enough to study mechanism. It is not enough to settle deployment behaviour for much larger closed models.

Third, adaptive attackers are not fully explored. A distiller could filter traces, mix data sources, query multiple times, train against suspected perturbations, or use stronger student objectives. The paper shows a defence under plausible conditions, not the end of the arms race. The title of this article was not chosen by accident.

Fourth, user-facing quality needs more than final-answer accuracy. If antidistillation inserts strange intermediate tokens while preserving the final answer, automated benchmarks may approve while customers raise eyebrows. The enterprise buyer’s tolerance for “technically correct but weirdly haunted” reasoning is, historically, limited.

The real lesson: reasoning traces need rights management

The paper’s deepest business implication is that reasoning traces should be treated like a licensed asset, not exhaust.

In software, source code, binaries, logs, telemetry, and API outputs have different access rules. In AI products, those distinctions are still immature. A reasoning trace can be simultaneously an explanation, a debugging object, a compliance artefact, a user feature, and a distillation dataset. That multi-use character makes it strategically dangerous.

Antidistillation sampling does not remove the danger. It gives model owners a mechanism for pricing it into generation.

That is the right frame. The future of model protection will not be a single wall around the weights. It will be layered: access controls, monitoring, contractual terms, watermarking, output filtering, trace policy, and decoding-time defences. Some layers will fail. Some will annoy users. Some will be bypassed. Defence is not a cathedral; it is plumbing with adversaries.

The paper contributes one important pipe.

Conclusion: the model’s answer is now part of the moat

Antidistillation sampling is a useful reminder that LLM competition is no longer just about who has the best model. It is also about who controls the learning value of the model’s outputs.

The old assumption was simple: if a model gives a better answer, the product is better. Reasoning models complicate that. A better answer with a richer trace may also be a better training example for someone else. The model is not only serving the customer; it may be tutoring its replacement.

The paper’s proposal is elegant because it intervenes at the moment of speech. It does not demand that providers abandon reasoning traces. It asks whether traces can be generated in a way that remains useful to the user but hostile to unauthorised distillation.

That is a practical and uncomfortable idea. Which is usually where the useful ones live.

Cognaptus: Automate the Present, Incubate the Future. :::