Integrated Gradients

TL;DR for operators A team using an importance-ranking tool must know whether its low-ranked model components are genuinely safe to remove—not merely whether the tool produces the same ranking repeatedly. On LLaMA-3.1-8B at 30% masking, Wanda achieved near-perfect ranking stability of 0.9998, yet removing the components it labelled least important drove perplexity to 360,083. Consensus-2 was less stable at 0.945, but the same removal test produced perplexity of 66.1. The more reproducible ranking was far less causally faithful. ...

TL;DR for operators GRAINS is not “fine-tuning, but cheaper.” That framing misses the point and commits the usual business sin of turning a mechanism into a procurement slogan. The paper’s useful claim is more specific: token-level attribution can be converted into an inference-time steering signal. Instead of retraining model weights, GrAInS identifies which text or image tokens most strongly push the model toward preferred or dispreferred outputs, builds layer-wise steering vectors from those activation shifts, and applies normalized edits during inference.1 ...