Model Pruning

TL;DR for operators A team using an importance-ranking tool must know whether its low-ranked model components are genuinely safe to remove—not merely whether the tool produces the same ranking repeatedly. On LLaMA-3.1-8B at 30% masking, Wanda achieved near-perfect ranking stability of 0.9998, yet removing the components it labelled least important drove perplexity to 360,083. Consensus-2 was less stable at 0.945, but the same removal test produced perplexity of 66.1. The more reproducible ranking was far less causally faithful. ...

A model can be small and still be badly organized. That is the quiet problem behind a lot of model compression work. We often ask whether a neural network can be pruned without losing too much accuracy. Fair enough. Budgets are real. Memory is not decorative. But the question hides a stronger assumption: that one sparse structure should serve every input equally well. ...