TL;DR for operators When an autonomous robot brakes, reroutes, or stops near a person, the operational question after an incident is larger than “which input features influenced the model?” Investigators may need the supporting sensor evidence, the state the system inferred, the alternatives it considered, why one action won, and whether execution matched the intended command. ...
TL;DR for operators A manipulation agent can reach the requested final state and still have executed the task unsafely. That creates a measurement problem for teams using task-completion rates to decide whether an embodied model, prompt, or policy update is ready for deployment. SafeRelBench tests this gap directly. Across seven evaluated VLM-driven agents, the spatial-relation cases produced task Success Rates (SR) of 0.52–0.73 but Safety Success Rates (SSR) of only 0.16–0.40. In matched non-spatial settings, SR rose to 0.83–0.94 and SSR reached as high as 0.91.1 The benchmark therefore measures something final-state success can miss: whether the agent satisfied the relevant safety prerequisite before taking the risky action. ...