Attribution

TL;DR for operators A RAG system can return the right answer, attach a source that genuinely supports that answer, and still leave one critical question unresolved: did that source actually influence the model’s answer generation? A mechanistic study of Llama-3.1-8B-Instruct finds that inline citation behavior is not controlled by one dedicated citation feature. It emerges from a distributed sequence of attention heads and MLPs that includes early entity enrichment, matching between document and question entities, mid-layer processing, and late aggregation that shifts the model toward emitting a citation marker rather than ending the sentence.1 ...

TL;DR for operators A forecasting dashboard can correctly report that an earlier observation influenced a prediction and still give the wrong impression about how that influence enters the model. Amadeo Tunyi’s paper, The Failures of Marginal Influence-Based Attribution Methods for Global Time Series Explanations, argues that familiar scalar attribution methods cannot in general recover the model’s direct temporal dependency structure.1 Marginal methods can assign importance to an earlier variable whose influence is entirely mediated through a later, autocorrelated observation. Gradient methods can report sensitivity that exists only outside the support of the data the model actually sees. ...