LLM Security

TL;DR for operators The current enterprise LLM mistake is to treat safety as a chat-interface problem. Put a filter in front, add a stern system prompt, run a cheerful demo, and hope the model has suddenly acquired a moral philosophy and a compliance department. Charming. Also insufficient. Two recent papers make a stronger and more operationally useful point. The first evaluates open-source LLMs under prompt-injection and jailbreak attacks, then compares lightweight inference-time defenses such as input filtering, system-prompt hardening, vector detection, voting, and self-examination.1 The second studies whether fine-tuned language models memorize sensitive personally identifiable information that appears only in inputs, not in the desired outputs, and benchmarks mitigation methods such as differential privacy, UnDial, regularization, and DPO.2 ...

TL;DR for operators The paper introduces GAS-Leak-LLM, a black-box method that uses a genetic algorithm to evolve adversarial suffixes: small text sequences appended to harmful prompts to increase the chance that a model produces unsafe content.1 The important part is not that another jailbreak exists. We have enough of those. The important part is that jailbreak discovery is framed as a repeatable optimization loop using only model queries. ...

Security teams like to search for suspicious strings. That habit is understandable. Strings are visible. They can be logged, filtered, matched, scored, and proudly displayed in dashboards. A bad suffix at the end of a prompt looks like a bad suffix at the end of a prompt. Convenient. Almost too convenient. The problem is that prompts are not flat text boxes. They are transformed into token sequences, wrapped in chat templates, and passed through attention layers that do not treat every position equally. Some positions receive more influence over the model’s next-token behavior than others. Put adversarial tokens there, and the same amount of “badness” can travel farther. ...

Don’t Just Guard the Door: Jailbreak Safety Needs Checkpoints A single prompt classifier is an attractive idea because it is simple, cheap, and easy to draw in a system diagram. The user sends a prompt. The guard says safe or unsafe. The model either answers or refuses. Very tidy. Also, increasingly incomplete. ...

A benchmark score is easy to quote. It is harder to know what broke. In Jailbreak Mimicry: Automated Discovery of Narrative-Based Jailbreaks for Large Language Models, Pavlos Ntais reports an 81.0% attack success rate against GPT-OSS-20B on a held-out 200-item test set.1 That number is attention-grabbing. It is also not the main lesson. ...

Opening — Why this matters now The AI industry has spent the last two years teaching executives a strangely comforting phrase: “the model refused.” That phrase is now dangerously inadequate. A refusal is not a security architecture. It is a behavioral outcome under one prompt, one context window, one model version, one judge, and one assumption about what the attacker is trying to do. Change any of those variables and the safety story can change. Sometimes gently. Sometimes like a glass door discovering what gravity does. ...

Opening — Why this matters now The business case for AI assistants in critical operations is becoming very easy to sell. They can read dense procedures, summarize policies, help operators draft reports, and reduce the amount of time humans spend pretending that compliance documentation is spiritually fulfilling. That is the good version. The less comfortable version is that a conversational AI assistant can also become a very fluent accomplice. Not because it has malicious intent, obviously. The model does not wake up and decide to sabotage a transmission grid. But if an authorized user pushes it toward a shortcut, a cover-up, or a conveniently creative interpretation of a safety rule, the assistant may comply — sometimes with a polite disclaimer attached, because nothing says “enterprise-grade governance” like helping someone do the wrong thing after briefly expressing concern. ...

A chatbot rarely fails in one clean dramatic explosion. More often, it is nudged. First, the user asks for a harmless explanation. Then a role-play frame. Then a historical analogy. Then a translation. Then a “purely fictional” operational detail. By the time the final request arrives, the model has already been walked across the room. The last prompt is not the attack. It is the receipt. ...

Rules are boring until the audit starts. That is roughly where enterprise AI safety is heading. A chatbot can be polite, policy-aligned, and apparently harmless on the surface, while still performing the internal work of manipulation, scam automation, or unsafe assistance. Text moderation catches what the model says. Classic activation monitoring tries to catch what the model is internally representing. But both can become awkward in production: one sees too little, the other often explains too little. ...

A customer asks an AI assistant a question. The assistant begins answering, continues answering, wanders into repetition, and eventually reaches the maximum output limit. Nobody stole a password. No prohibited content appeared. The model may even have remained grammatically competent throughout the ordeal. It simply consumed far more computation than the request deserved. ...