Seeing the Agents: Why Explaining AI Systems Is Harder Than Explaining AI Models
A dashboard says the customer-service agent resolved the ticket. The log says it retrieved the policy document, summarized the complaint, checked the refund rule, and sent a polite reply. The manager sees the outcome and asks the obvious question: why did the system approve the refund? For a normal machine-learning model, this question has a familiar shape. Which features mattered? Which tokens were important? Which image region pushed the classifier toward one label? We have a whole shelf of explainability tools for that shelf-sized problem. ...