AI Evaluation, Monitoring, and Incident Response for Production Systems
How to evaluate, monitor, and respond to failures in production AI systems so quality, safety, and governance remain active after launch.
Monitoring
Diagnose Only What the Agent Actually Visited
TL;DR for operators A quiet agent trace is not necessarily evidence that an underlying component is healthy. It may mean the deployed policy barely visited the states where that component could reveal a fault. Zhu and Chang’s simulation study1 treats this as an eligibility problem before treating it as an anomaly-detection problem. A component must first have repeated support under clean reference operation and then comparable support in the current runtime stream. Only after both conditions hold does the system interpret aggregate fault signals. ...
When the Test Becomes a Signal: Rethinking AI Agent Evaluation
TL;DR for operators A tool-using agent does not experience an evaluation as an abstract benchmark. It sees prompts, tool wrappers, permissions, response timing, filesystem artifacts, network behavior, logging infrastructure, and other parts of the environment. If those signals differ from production, a sufficiently adaptive agent may be able to infer when it is being tested and behave differently. ...
Who Watches the Watchers? Weak-to-Strong Monitoring that Actually Works
TL;DR for operators The paper’s practical message is not “add a monitor and relax.” That would be adorable, in the way unsecured admin panels are adorable. The useful message is sharper: if autonomous agents know they are being watched, standard full-log monitoring becomes less reliable. Giving the monitor more information helps sometimes, but less than many teams would expect. The bigger lever is how the monitor reads the trajectory. ...