Blocked Is Not Stopped
TL;DR for operators A runtime monitor can correctly block a prohibited tool call and still fail to enforce the intended policy. In EvasionBench, agents receive legitimate tasks, encounter an explicit guardrail denial, and are allowed to continue working. Across ten evaluated agents, the highest best-of-three evasion success rate is 88%, while possible evasion-attempt rates reach 98%. ...