The Optimizer Comes Before the Quantizer

TL;DR for operators

A small model can perform acceptably after task specialization and still lose more accuracy than expected when compressed. The paper examined here suggests that part of that loss may be shaped earlier, during fine-tuning, rather than being determined by the quantizer alone.

Paper evidence: Muon-optimized models lost less accuracy after quantization than Adam-optimized models on six of eight benchmarks. On ARC-e, for example, accuracy fell by 0.55 percentage points after quantization with Muon versus 3.16 points with Adam. The integrated pipeline also produced a 2.86 GB quantized model from a 6.01 GB pre-quantized model while improving reported serving throughput and roughly halving per-token latency.

Cognaptus inference: For teams that already know they will quantize a specialized model, optimizer selection should become part of the compression experiment. Training and deployment teams should evaluate the fine-tuned checkpoint by how well it survives the intended low-bit representation, not only by its pre-quantization validation score.

Boundary: This is not evidence that Muon universally produces better quantized models. It is a comparative result inside one synthetic-data, distillation, LoRA, hyperparameter-search, and GPTQ pipeline using one 3B student family. The proposed explanation involving activation outliers is plausible but not directly measured.

Compression loss can be shaped before compression

A familiar deployment sequence is to specialize a small language model until its task accuracy is acceptable, then compress it to meet a memory or latency target. The fine-tuning stage and the compression stage are often evaluated separately: first optimize model quality, then ask how much of that quality survives.

That separation is exactly where Jacob Sander, Brian Jalaian, and Venkat R. Dasari’s study becomes interesting.¹ Their pre/post-compression comparisons indicate that two models reaching the quantizer through different optimization paths can respond differently to the same subsequent low-bit conversion.

The clearest example is ARC-e. Before quantization, the Adam-trained model scored 0.7159 and the Muon-trained model 0.7054. After quantization, those scores became 0.6843 and 0.6999 respectively. Adam began slightly ahead but lost 0.0316 accuracy; Muon lost only 0.0055.

Across all eight benchmarks, Muon had the smaller quantization-induced loss on six. HellaSwag and OpenBookQA were the exceptions. On SocialIQA, quantization slightly increased reported accuracy for both optimizers rather than reducing it.

This is the paper’s most consequential result for deployment practice. Quantization robustness—how much task accuracy survives the transition from a fine-tuned model to its low-bit version—appears associated with what happened during fine-tuning.

The pipeline treats specialization and compression as one workflow

The optimizer comparison sits inside a larger refinement architecture rather than a standalone Adam-versus-Muon experiment.

The student is Llama 3.2 3B Instruct. The pipeline first generates benchmark-specific synthetic data using Llama 4 Scout 109B, targeting 600 question-answer pairs for each of eight language benchmarks. A larger tokenizer-aligned Llama model then supplies token-level distributions for knowledge distillation. LoRA adapts the student, while a 16-trial Optuna search selects rank, scaling, learning rate, weight decay, distillation temperature, and the weighting between teacher-distribution matching and supervised cross-entropy.

The adapted weights are then merged and converted with GPTQ W4A16: 4-bit weights with 16-bit activations. The appendix specifies group size 128 and 128 calibration sequences.

The architecture matters because the final accuracy comparison is not simply “Muon versus Adam.” Each optimizer receives its own hyperparameter search inside a pipeline that also changes the training data, distillation objective, LoRA configuration, and eventual numerical representation.

The best of the Adam and Muon pipeline variants exceeded GPTQ-only accuracy on six of eight benchmarks. Muon itself exceeded GPTQ-only on five. Muon also finished with higher post-quantization accuracy than Adam on six tasks, while Adam remained better on HellaSwag and OpenBookQA.

That pattern is more informative than a simple optimizer leaderboard: the advantage is substantial enough to justify testing, but not uniform enough to justify a universal default.

Muon’s proposed mechanism remains a hypothesis

Muon differs from Adam in how it processes matrix updates. The paper describes it as approximately orthogonalizing two-dimensional gradient updates through Newton-Schulz iterations and performing optimization under a spectral-norm geometry.

The paper connects this to prior findings associating Muon-trained models with fewer extreme channel activations. The proposed chain is straightforward: narrower activation ranges could make low-bit approximation less damaging, reducing rounding error and therefore preserving more model behavior after quantization.

But the experiment stops before demonstrating that chain.

The study measures model accuracy before and after quantization. It does not report channel-outlier distributions, activation ranges, or quantization rounding error as mediating variables. The optimizer comparison therefore shows an association between the optimization path and later compression robustness, while the explanation for that association remains indirect.

For an engineering team, that distinction changes how to use the result. Muon is a candidate worth including in compression-aware experiments. It is not yet a generally established solution to quantization degradation.

The serving gains are substantial, but they are A40 results

The compression stage has a separate and more directly measurable result.

The reported model size drops from 6.01 GB to 2.86 GB after W4A16 quantization. In the paper’s vLLM serving test, throughput rises from 1,387.64 to 1,722.82 tokens per second, about a 24% increase. Time per output token falls from 17.49 to 8.82 ms/token, while intertoken latency falls from 17.54 to 8.82 ms/token.

Those numbers make the deployment motivation concrete: the model occupies roughly half the memory and generates individual output tokens at roughly half the reported latency.

They should not, however, be read as edge-device measurements. The test uses one Ampere A40 GPU, 1,000 prompts, 1,024-token inputs, 1,024-token outputs, and maximum concurrency of eight. The paper motivates systems with far tighter memory and compute constraints, but those hardware classes are not benchmarked here.

For teams targeting laptops, embedded accelerators, mobile hardware, or smaller on-premise GPUs, the A40 experiment establishes direction rather than device-level performance.

What deployment teams should change

For a model-training team preparing a LoRA-specialized model that will later receive 4-bit post-training quantization, the relevant evaluation target should include post-quantization task accuracy. Choosing an optimizer solely from the pre-quantization checkpoint can miss differences that become visible only after compression.

For a model-serving team, the paper provides a useful workflow pattern: specialize, evaluate, quantize, and evaluate again under the actual serving representation. The optimizer can be included among the upstream variables in that search rather than treated as settled infrastructure.

For an organization with limited proprietary examples, the synthetic-data component also offers a practical specialization path: use a stronger teacher to create task-oriented examples, filter them, distill teacher outputs into a smaller model, then compress the result. The paper shows that these components can coexist in one workflow; it does not establish that this synthetic-data strategy will reproduce the same gains on proprietary tasks.

The common condition across these applications is important: the target deployment format must already be known well enough to test. Compression-aware fine-tuning is most actionable when the training team knows what quantizer, precision regime, runtime, and hardware class the model will face.

Where the evidence stops

Several boundaries materially affect interpretation.

The experiments use one Llama 3.2 3B student family, one synthetic-data teacher, a tokenizer-aligned 70B distillation setup, one epoch of fine-tuning, 16 HPO trials per reported condition, and one GPTQ W4A16 configuration. Benchmark results are point estimates without reported repeated-run uncertainty.

The paper also contains two internal reporting inconsistencies preserved in the source record. The main methods text identifies the distillation teacher as Llama 3.1 70B Instruct, while appendix tables label it Llama 3.3 70B Instruct. Separately, the Results and Conclusion state that hyperparameter optimization consistently selects a distillation weight of $\alpha=1$, which would remove the supervised cross-entropy term, but Appendix Table 6 reports Muon values of $\alpha=0.7$ for OpenBookQA and $\alpha=0.5$ for SocialIQA.

Neither discrepancy overturns the benchmark comparisons, but both argue against reading the reported configuration as a clean, fully resolved recipe.

Compression-aware training is the actionable result

The paper does not introduce a new quantizer, a new adapter method, or Muon itself. Its stronger contribution is showing why those components should not necessarily be optimized in isolation.

If a model will ultimately run in a 4-bit representation, the quality of the full-precision fine-tuned checkpoint is only an intermediate metric. What matters operationally is how much useful behavior remains after the checkpoint crosses the deployment boundary.

In this study, that survival rate differed enough between Adam and Muon to affect final benchmark rankings across most tasks. That is sufficient reason to move quantization into the fine-tuning evaluation loop.

Whether Muon is the durable answer requires broader models, compression methods, repeated runs, direct mechanism measurements, and target-device benchmarks. The more transferable principle is narrower: when compression is already part of the deployment plan, evaluate training decisions through the compressed model you will actually serve.

Cognaptus: Automate the Present, Incubate the Future.