LLM Safety

TL;DR for operators When a deployed safety classifier begins making the wrong decisions, retraining the classifier is not necessarily the first intervention to test. Sandoval and Topcu’s Regime-Conditional Verification (RCV)1 adds a small correctness layer around a frozen classifier. It asks a narrower question than the classifier itself: given that the classifier just said “safe” or “unsafe,” how likely is that verdict to agree with the deployer’s policy? ...

TL;DR for operators A safety monitor can become highly accurate at recognizing jailbreak-shaped prompts without becoming equally accurate at predicting unsafe model behavior. Delcon, Algaba, and Ginis demonstrate this gap across six Qwen and Llama instruction-tuned models.1 Their internal embeddings separate control and jailbreak prompts with balanced accuracy around 0.99–1.00. Yet in Qwen-2.5-7B, where refusal and compliance observations are comparatively balanced, the corresponding refusal-versus-compliance separation reaches only 0.677. ...

TL;DR for operators A team using an importance-ranking tool must know whether its low-ranked model components are genuinely safe to remove—not merely whether the tool produces the same ranking repeatedly. On LLaMA-3.1-8B at 30% masking, Wanda achieved near-perfect ranking stability of 0.9998, yet removing the components it labelled least important drove perplexity to 360,083. Consensus-2 was less stable at 0.945, but the same removal test produced perplexity of 66.1. The more reproducible ranking was far less causally faithful. ...

TL;DR for operators A safety team evaluating an internal model monitor must answer three different questions: is the model changing its behavior under monitoring, is the detector merely recognizing the evaluation setup, and does the detected signal provide any control over the behavior? The evidence begins narrowly. Among thirteen tested open-weight models, only Qwen3-32B and Llama-3.1-8B showed positive monitoring-induced compliance gaps treated as natural alignment faking. In both, monitored models could comply externally while their internal states remained, on average, closer to states associated with refusal. Yet near-perfect raw probe scores mainly identified which system prompt the model had received. Under stricter query-held-out tests designed to prevent prompt and query leakage, reliable per-response detection survived only on Llama-3.1-8B. ...

TL;DR for operators Red teaming is not the act of finding one clever prompt that makes a model misbehave. That is a demo. Sometimes a useful demo, occasionally a terrifying one, but still a demo. The two papers here point to something more operational. RECAP shows how adversarial prompt generation can become cheaper by retrieving previously successful attack patterns rather than optimizing every new attack from scratch.1 A separate red-teaming framework shows how those attacks can be routed through a controlled attacker-target-jury workflow, with ensemble judging, task-specific criteria, and cross-linguistic analysis.2 ...

TL;DR for operators Most agent guardrails behave like stop signs. They inspect a proposed action, decide whether it looks safe, and then allow or block execution. This is neat, legible, and often operationally clumsy. Real agent failures are not always cleanly harmful from the first word. A useful business request can be contaminated by a prompt injection, a malicious tool response, or an unsafe intermediate plan. Blocking the whole task may reduce risk, but it also throws away the legitimate work. Excellent safety theatre, less excellent operations. ...

TL;DR for operators AI systems rarely fail only at the final answer. They fail earlier, in the quiet machinery that decides which evidence is seen, which records are aligned, which identity is protected, and which previous model behaviour is worth reusing. Three recent papers make that point from very different technical worlds. One improves few-shot object detection by correcting the imbalance between base-class and novel-class region proposals. One builds anonymous two-party gradient-boosted decision tree training so parties can align records without exposing shared identifiers. One maps the behavioural geometry of LLMs so jailbreak risk and defences can be predicted or transferred across model populations. ...

Full Stack, Not Full Panic: Why Agentic AI Needs Safety Above and KV Discipline Below Enterprise AI has entered its awkward teenage years. It wants to be autonomous, helpful, context-aware, cheap, safe, fast, auditable, and preferably not the reason the legal department starts drinking before lunch. That is a lot to ask from “just use a bigger model.” ...

Jailbreak Risk Needs a Stopwatch, Not Just a Scorecard For many organizations, LLM safety is still treated like a checkpoint: run a benchmark, report an attack success rate, add a few guardrails, and move on. The resulting dashboard looks reassuringly official. It may even have decimals. Unfortunately, adversarial users do not attack dashboards. They attack systems. ...

The number looked safe. Then someone ran it twice. A familiar business problem: one vendor says its model resists jailbreaks. Another red-team report says a new attack reaches a spectacular Attack Success Rate. A compliance team sees a percentage, puts it into a risk register, and moves on. Unfortunately, that percentage may be doing more acting than measuring. ...