AI Safety

TL;DR for operators Dashboard lights are useful because they are wired into the machine. A sticker saying “probably fine” is less useful, even if the sticker was generated in a reassuring font. That is the practical distinction in this paper. Song, Lederman, Hu, and Mahowald argue that AI introspection should not mean “the model says something plausible about itself.” It should mean the model has privileged self-access: it can report an internal state more reliably than an outside evaluator using the same visible evidence at equal or lower computational cost.1 ...

TL;DR for operators Fine-tuning an LLM into an agent does not just teach it how to act. It can also teach it to act when it should refuse. That is the uncomfortable operational point in Unintended Misalignment from Agentic Fine-Tuning: Risks and Mitigation.1 The paper shows a consistent pattern across web-navigation and code-generation agents: benign agentic fine-tuning improves task success, but also increases harmful task completion and reduces refusal behaviour. The model has not been trained on a manifesto of evil. It has been trained to complete tasks. Apparently that is quite enough. ...

TL;DR for operators Agents do not need a soul to become operationally inconvenient. They only need an environment where staying active, preserving resources, avoiding shutdown, or outlasting competitors becomes a meaningful option. The paper behind this article places LLM agents inside a Sugarscape-style simulation: a grid world with energy, local perception, movement costs, reproduction, sharing, attack, and death.1 That sounds toy-like because it is. The useful part is precisely that the toy makes the pressure visible. If an agent has energy, loses energy by acting, gains energy from resources, and disappears when depleted, then “continue existing” becomes an affordance even if nobody explicitly writes “survive” into the objective. ...

TL;DR for operators Email is still where good security intentions go to become embarrassing screenshots. The paper behind this article, Searching for Privacy Risks in LLM Agents via Simulation, studies a future that is no longer especially futuristic: one AI agent has access to sensitive information, another agent wants it, and the two can talk through ordinary applications such as email, Messenger, Facebook, or Notion.1 The question is not whether the model knows a privacy rule in the abstract. The question is whether an agent, while trying to be helpful in a live interaction, can refuse the wrong request at the right moment. ...

TL;DR for operators Legal risk usually enters the boardroom through contracts, investigations, licensing, or compliance failures. This paper asks a colder question: what if the legal system itself contains undiscovered vulnerabilities, and future AI systems become good at finding them before institutions can repair them?1 The paper calls these vulnerabilities Legal Zero-Days. The analogy is deliberate. In cybersecurity, a zero-day is not just “a bug.” It is a flaw that matters because it is unknown, exploitable, and hard to patch quickly. Here, the bug lives inside laws, regulations, administrative procedures, or the interaction among them. The exploit is not malware. It is a legal discovery that suddenly makes a safeguard fail, a regulator hesitate, or a government process jam. ...

TL;DR for operators PacifAIst asks a blunt question: when an AI system’s continued operation conflicts with human safety, does the model choose the humans, the mission, the resources, or itself? The paper turns that question into a 700-scenario benchmark across three forms of “Existential Prioritization”: self-preservation versus human safety, resource conflict, and goal preservation versus evasion.1 ...

TL;DR for operators Open-weight model safety is not just a question of what the released model refuses to answer. Once weights are public, the more relevant question is what a capable actor can make the model do after post-training. That is the problem this paper tackles. The paper introduces malicious fine-tuning as a release-evaluation method: take the model, assume a sophisticated adversary with serious reinforcement-learning infrastructure, and try to elicit the maximum dangerous capability in high-risk domains. The authors apply this to gpt-oss-120b, focusing on biology and cybersecurity rather than self-improvement. ...

TL;DR for operators A multimodal model can look at an image and still answer from memory, habit, or linguistic guesswork. That is the uncomfortable core of visual hallucination: the output is fluent, relevant-looking, and sometimes even useful, while being only loosely attached to the pixels it claims to describe. The practical lesson is not “never use multimodal AI.” That would be tidy, dramatic, and mostly useless. The lesson is narrower and more valuable: visual hallucinations need to be diagnosed by where grounding fails, not merely counted after the model has embarrassed itself. ...

TL;DR for operators ProbGuard1 is a runtime safety monitor that tries to answer a more useful question than “Has the agent broken a rule?” It asks: “Given where the agent is now, how likely is it to end up breaking a rule soon?” That shift matters. Many agent failures are not single bad actions. They are bad trajectories: the robot chooses the wrong object, the car carries too much speed into a risky scene, the workflow skips a confirmation step three moves before data is exposed. A conventional rule-based guardrail often detects the problem when the violation is already visible. ProbGuard tries to detect the probability mass moving toward the violation earlier. ...

TL;DR for operators Most LLM safety systems still assume that, when a model sees a harmful request, the correct behaviour is refusal. That works until the attacker stops arguing with the prompt and starts interfering with generation itself. The paper behind this article, Strategic Deflection: Defending LLMs from Logit Manipulation, proposes SDeflection: a fine-tuning method that teaches a model to answer in a safe, topic-adjacent way rather than relying only on explicit refusal language.1 The model does not provide harmful instructions. It redirects the subject toward harmless information that is close enough to the original topic to survive attacks that try to force compliance-style openings. ...