Important, but Not Direct: When Time-Series Attribution Misstates Model Dependencies
TL;DR for operators A forecasting dashboard can correctly report that an earlier observation influenced a prediction and still give the wrong impression about how that influence enters the model. Amadeo Tunyi’s paper, The Failures of Marginal Influence-Based Attribution Methods for Global Time Series Explanations, argues that familiar scalar attribution methods cannot in general recover the model’s direct temporal dependency structure.1 Marginal methods can assign importance to an earlier variable whose influence is entirely mediated through a later, autocorrelated observation. Gradient methods can report sensitivity that exists only outside the support of the data the model actually sees. ...