Runtime Monitoring

TL;DR for operators Adding a second model does not automatically make an AI workflow safer. It creates another opportunity to correct an error—and another opportunity to introduce one. In the paper’s cybersecurity experiment, giving Gemma-2’s reasoning to Phi-3 raises Phi-3’s accuracy from 60.34% to 93.10%. In networking, the direction reverses for the stronger model: Gemma-2 falls from 90.82% to 89.80% after reasoning exchange. Passing the outputs to a Llama 3.2 judge reduces networking accuracy further, to 88.78%. ...

Safety by Design, Rewritten: When Data Defines the Boundary Boundaries are usually drawn before deployment. A product team defines where a system is allowed to operate, safety engineers translate that into requirements, regulators ask whether the evidence matches the claim, and everyone pretends the world politely fits inside the diagram. Charming. Occasionally even useful. ...

Guardrails usually fail quietly. A user sends a malicious prompt. The model begins answering. The safety policy that looked firm in the demo environment starts behaving like office wallpaper: present, decorative, and not especially involved. By the time a post-hoc filter reads the final answer, the model has already produced the thing it should not have produced. The system may block the response from the user, but the real lesson is less flattering: the model crossed the line before the defense noticed. ...

TL;DR for operators ProbGuard1 is a runtime safety monitor that tries to answer a more useful question than “Has the agent broken a rule?” It asks: “Given where the agent is now, how likely is it to end up breaking a rule soon?” That shift matters. Many agent failures are not single bad actions. They are bad trajectories: the robot chooses the wrong object, the car carries too much speed into a risky scene, the workflow skips a confirmation step three moves before data is exposed. A conventional rule-based guardrail often detects the problem when the violation is already visible. ProbGuard tries to detect the probability mass moving toward the violation earlier. ...