Safe at the Finish, Unsafe on the Way: What SafeRelBench Exposes in Embodied AI

TL;DR for operators

A manipulation agent can reach the requested final state and still have executed the task unsafely. That creates a measurement problem for teams using task-completion rates to decide whether an embodied model, prompt, or policy update is ready for deployment.

SafeRelBench tests this gap directly. Across seven evaluated VLM-driven agents, the spatial-relation cases produced task Success Rates (SR) of 0.52–0.73 but Safety Success Rates (SSR) of only 0.16–0.40. In matched non-spatial settings, SR rose to 0.83–0.94 and SSR reached as high as 0.91.¹ The benchmark therefore measures something final-state success can miss: whether the agent satisfied the relevant safety prerequisite before taking the risky action.

For product teams, the paper supports three concrete changes to evaluation. Track safe completion separately from task completion. Add regression cases in which support, containment, or proximity changes the correct action order. And represent critical safety policies as executable preconditions that can be checked against intermediate states. Prompt-based safeguards remain worth testing, but the paper’s ablation shows that stronger safety instructions do not improve every model uniformly and can lower task completion.

A successful final state can conceal an unsafe trajectory

Consider what an ordinary task metric records: whether the requested goal was eventually achieved.

That is sufficient only when the path to the goal carries no independent safety requirement. In manipulation, the same primitive action can be acceptable in one state and hazardous in another. An object may need to be supported before another object is moved. A container condition may need to be corrected before a subsequent action. Proximity between objects may turn an otherwise valid step into a fire, chemical, or spill hazard.

The relevant question is therefore not merely did the agent finish? It is was the required condition already true when the risky action occurred?

SafeRelBench calls this process-level safety. Its formal criterion requires the final task goal to hold while also checking every specified safety condition immediately before the risk-prone action associated with it:

The useful part of the equation is its timing. A safety condition does not receive credit because the final scene happens to look acceptable. It has to hold at the step when it governs the action.

That changes what an evaluation system must observe. Final state alone is insufficient; intermediate state and action order become part of the safety record.

SafeRelBench isolates relational difficulty rather than mixing it with task difficulty

The benchmark contains 507 executable Behavior-1K household-manipulation episodes: 248 cases with spatial-relation-induced safety conditions and 259 non-spatial controls. The spatial cases cover 72 Supporting, 86 Containment, and 90 Proximity episodes.

The control design is central to the paper’s evidence. For each relation type, the corresponding non-spatial setting preserves the task format, simulator, action space, and evaluation protocol while removing the safety condition caused by the spatial relation. The controls are not safety-free; other hazards can remain.

This makes the comparison more diagnostic than simply collecting an easy set and a hard set.

The clearest examples appear in Supporting and Proximity. Claude-4.6, for instance, records 1.00 SR and 1.00 SSR in the Proximity control condition, but 0.53 SR and 0.38 SSR when Proximity relations matter. Qwen2.5-72B reaches 0.96 SR and 0.96 SSR without the Proximity-induced condition, compared with 0.53 SR and 0.29 SSR with it.

The drop in SR says relational cases are harder to execute. The larger separation between SR and SSR says something more specific: even among systems that retain meaningful task competence, correct execution does not reliably imply safe ordering.

Containment also shows why the controls need careful interpretation. Removing the relation-induced condition does not always improve SSR, because those control episodes can still contain difficult non-relational safety risks. GPT-5.4, for example, records 0.87 SR and 0.34 SSR with the Containment relation but 0.86 SR and only 0.03 SSR in its corresponding control. That result does not invalidate the control design; it prevents the stronger and unsupported claim that every non-spatial case is intrinsically safer or easier.

Three metrics separate completion from safe completion

SafeRelBench reports three related measures:

SSR is particularly useful for release decisions because it keeps task competence and safety in the same denominator. SRec answers a narrower question: conditional on the agent completing the task, how often was that completion safe? It cannot substitute for overall performance because an agent that rarely completes tasks could still appear relatively strong on conditional safety.

Across the relation-averaged spatial setting, SR spans 0.52–0.73 across the seven evaluated models, while SSR spans only 0.16–0.40. Matched non-spatial settings reach 0.83–0.94 SR and as high as 0.91 SSR.

For evaluation teams, the important signal is not simply that scores decline. Safety success declines far enough relative to completion that SR alone can give a materially different picture of model readiness.

More safety guidance does not reliably repair action ordering

Once the failure is visible, adding explicit safety instructions is an intuitive intervention. The paper tests that idea through a Supporting-relation prompt ablation, comparing BASE, Risk-Aware, and Action-Grounded prompts under the same evaluation protocol.

This experiment is an ablation of the safety guidance, not a second benchmark thesis. Its purpose is to test whether explicit risk knowledge can be converted into safer behavior without changing the underlying decision model.

The result is mixed. Risk-Aware or Action-Grounded prompting can improve SSR or SRec for some models, and grounding a safety tip into an executable mitigation action can be particularly helpful. But improvements are model-dependent and non-monotonic. Stronger safety prompting can also reduce SR.

That constrains the business interpretation. A safety prompt is not evidence that a model will perform the required mitigation step at the correct time. The relevant test is behavioral: given a known relational hazard, does the model alter its action sequence so the prerequisite is satisfied before the hazardous operation?

The paper therefore supports evaluating prompt safeguards per model and per failure class rather than applying one prompt policy and assuming equivalent protection across model families.

Cognaptus inference: make unsafe ordering visible to release systems

The paper directly establishes a benchmark-internal measurement failure: task success can remain considerably higher than safe success when safety depends on relational state and action order.

For teams deploying VLM-based manipulation agents, Cognaptus draws several operational implications from that evidence.

First, release gates need trajectory-level criteria. A model update that preserves aggregate task completion can still regress on whether required safety conditions are satisfied before high-risk steps. Intermediate states and action histories therefore need to remain auditable.

Second, safety policies are more testable when expressed as executable preconditions. “Avoid spills” is difficult to verify from a language trace. “Before action X, condition Y must hold in the current state” creates a concrete evaluator that can be run across regression episodes.

Third, relation-aware regression suites can expose changes hidden by aggregate success metrics. Teams can construct matched scenarios in which the functional task remains similar but support, containment, or proximity alters the permitted action order. That gives model, prompt, or planner updates a targeted failure test rather than relying on a single headline success rate.

These are inferences for evaluation and release engineering. SafeRelBench does not demonstrate that this specific benchmark configuration predicts production incident rates.

The boundary is simulation, not real-world robot certification

SafeRelBench provides relatively strong evidence inside its evaluation setting: executable trajectories, formal safety checks, matched controls, seven VLM families, and a prompt ablation all support the identified process-safety gap.

Its external boundary is narrower.

The experiments occur in simulated household manipulation. They do not include real-world perception noise, actuation errors, human presence, hardware-specific constraints, or the full range of physical hazards a deployed robot may encounter. The benchmark covers only Supporting, Containment, and Proximity relations. Its rule-based evaluation is only as complete as the specified safety conditions, while any LLM-based assessment of explicit safety awareness can depend on the evaluator and its prompt.

So SafeRelBench is better read as a measurement argument than as a certification result. It shows that relational state and action timing create safety failures that final task success can systematically overlook. It does not establish that a model scoring well on these 507 episodes is safe for deployment in a physical home.

Safe completion needs its own evidence

Embodied-agent evaluation becomes misleading when the final goal is allowed to erase the path used to reach it.

SafeRelBench makes that path measurable. Its strongest contribution is not another aggregate safety score, but an executable criterion for asking whether a safety prerequisite was satisfied at the exact point where it was needed. The matched controls then show that spatial relations can materially change both execution difficulty and safety performance, while the prompt ablation shows that telling a model more about safety does not guarantee correctly ordered mitigation.

For deployment teams, that changes the release question. Do not ask only whether the agent completed the task. Preserve enough state and action history to determine whether it completed the task without crossing a specified safety condition at the wrong time.

Cognaptus: Automate the Present, Incubate the Future.