The Safety Shield That Must Account for Its Risk
TL;DR for operators An autonomous agent can improve its task performance by taking actions that increase the probability of violating an operating constraint. RecReg addresses this through a separate recovery policy and an action-level shield that evaluates future violation risk before allowing an action to execute. The framework offers conditional mathematical safety guarantees in finite tabular environments, plus simulated evidence of reduced violations in deep reinforcement learning. In SafetyGym PointGoal1, its deep implementation reduces the reported violation rate from 0.0562 to 0.00500 relative to unconstrained DreamerV3, while average task return falls from 22.2 to 17.3. ...