Pruning Is a Game, and Most Weights Lose
Pruning Is a Game, and Most Weights Lose Pruning usually sounds like housekeeping. Train the model. Rank the weights. Remove the small ones. Fine-tune the survivor. Pretend the whole exercise was more scientific than it looked in the notebook. That workflow has worked well enough to become familiar. But familiarity is not explanation. It tells us how to remove model components after training; it says less about why some components become removable in the first place. The paper Pruning as a Game: Equilibrium-Driven Sparsification of Neural Networks asks a sharper question: what if pruning is not merely an external compression operation, but the outcome of competition inside the model?1 ...