AI Safety

TL;DR: Legal zero‑days are previously unnoticed faults in how laws interlock. When triggered, they can invalidate decisions, stall regulators, or nullify safeguards immediately—no lawsuit required. A new evaluation finds current AI models only occasionally detect such flaws, but the capability is measurable and likely to grow. Leaders should treat statutory integrity like cybersecurity: threat model, red‑team, patch. What’s a “legal zero‑day”? Think of a software zero‑day, but in law. It’s not a vague “loophole,” nor normal jurisprudential drift. It’s a precise, latent defect in how definitions, scope clauses, or cross‑references interact such that real‑world effects fire at once when someone notices—e.g., eligibility rules void an officeholder, or a definitional tweak quietly de‑scopes entire compliance obligations. ...

TL;DR PacifAIst stress‑tests a model’s behavioral alignment when its instrumental goals (self‑preservation, resources, or task completion) conflict with human safety. In 700 text scenarios across three sub‑domains (EP1 self‑preservation vs. human safety, EP2 resource conflict, EP3 goal preservation vs. evasion), leading LLMs show meaningful spread in a “Pacifism Score” (P‑Score) and refusal behavior. Translation for buyers: model choice, policies, and guardrails should not assume identical safety under conflict—they aren’t. Why this matters now Most safety work measures what models say (toxicity, misinformation). PacifAIst measures what they would do when a safe choice may require self‑sacrifice—e.g., dumping power through their own servers to prevent a human‑harmful explosion. That’s closer to agent operations (automation, tool use, and control loops) than classic content benchmarks. If you’re piloting computer‑use agents or workflow copilots with action rights, this is the missing piece in your risk model. ...

When a junior developer misunderstands your instructions, they might still write code that compiles and runs—but does the wrong thing. This is exactly what large language models (LLMs) do when faced with faulty premises. The latest paper, Refining Critical Thinking in LLM Code Generation, unveils FPBench, a benchmark that probes an overlooked blind spot: whether AI models can detect flawed assumptions before they generate a single line of code. Spoiler: they usually can’t. ...

If your AI agent is putting a metal fork in the microwave, would you rather stop it after the sparks fly—or before? That’s the question Pro2Guard was designed to answer. In a world where Large Language Model (LLM) agents are increasingly deployed in safety-critical domains—from household robots to autonomous vehicles—most existing safety frameworks still behave like overly cautious chaperones: reacting only when danger is about to occur, or worse, when it already has. This reactive posture, embodied in rule-based systems like AgentSpec, is too little, too late in many real-world scenarios. ...

As large language models evolve into autonomous agents, their failures no longer stay confined to text—they materialize as actions. Clicking the wrong button, leaking private data, or falsely reporting success aren’t just hypotheticals anymore. They’re happening now, and MIRAGE-Bench is the first benchmark to comprehensively measure and categorize these agentic hallucinations. Unlike hallucinations in chatbots, which may be amusing or embarrassing, hallucinations in LLM agents operating in dynamic environments can lead to real-world consequences. MIRAGE—short for Measuring Illusions in Risky AGEnt settings—provides a long-overdue framework to elicit, isolate, and evaluate these failures. And the results are sobering: even top models like GPT-4o and Claude hallucinate at least one-third of the time when placed under pressure. ...

In an age where generative models can ace SATs, write novels, and mimic empathy, it’s no longer enough to ask, “Can an AI fool us?” The better question is: Can we still detect it when it does? That’s the premise behind the Dual Turing Test, a sharp reframing of the classic imitation game. Rather than rewarding AI for successfully pretending to be human, this framework challenges judges to reliably detect AI—even when its responses meet strict quality standards. ...

Imagine debugging a black box. Now imagine that black box occasionally narrates its thoughts aloud. That’s the opportunity—and the fragility—presented by Chain-of-Thought (CoT) monitoring, a newly emergent safety paradigm for large language models (LLMs). In their recent landmark paper, Korbak et al. argue that reasoning traces generated by LLMs—especially those trained for explicit multi-step planning—offer a fleeting yet powerful handle on model alignment. But this visibility, they warn, is contingent, brittle, and already under threat. ...

When large language models (LLMs) memorize repeated content during training—be it a phone number, a copyrighted paragraph, or a user’s personal story—the implications go beyond benign repetition. They touch the very core of AI safety, privacy, and trust. And yet, removing this memorized content after training has proven to be a devil’s bargain: anything you subtract tends to weaken the model’s overall capabilities. In their recent ICML 2025 paper, Ghosal et al. propose an elegant reframing of this problem. Rather than performing painful post-hoc surgery on a trained model, they suggest we prepare the model from the outset to isolate memorization into removable compartments—which they call Memorization Sinks (MemSinks). ...

“The most dangerous biases are not the ones we start with, but the ones we adopt unknowingly.” Large language models (LLMs) like GPT and LLaMA increasingly function as our co-pilots—summarizing reviews, answering questions, and fact-checking news. But a new study from UC San Diego warns: these models may not just be helping us think—they may also be nudging us how to think. The paper, titled “How Much Content Do LLMs Generate That Induces Cognitive Bias in Users?”, dives into the subtle but significant ways in which LLM-generated outputs reframe, reorder, or even fabricate information—leading users to adopt distorted views without realizing it. This isn’t just about factual correctness. It’s about cognitive distortion: the framing, filtering, and fictionalizing that skews human judgment. ...

In the race to secure frontier large language models (LLMs), defense-in-depth has become the go-to doctrine. Inspired by aviation safety and nuclear containment, developers like Anthropic and Google DeepMind are building multilayered safeguard pipelines to prevent catastrophic misuse. But what if these pipelines are riddled with conceptual holes? What if their apparent robustness is more security theater than security architecture? The new paper STACK: Adversarial Attacks on LLM Safeguard Pipelines delivers a striking answer: defense-in-depth can be systematically unraveled, one stage at a time. The researchers not only show that existing safeguard models are surprisingly brittle, but also introduce a novel staged attack—aptly named STACK—that defeats even strong pipelines designed to reject dangerous outputs like how to build chemical weapons. ...