Red Teaming

Security testing is not a screenshot. A model gives a dangerous answer. Someone posts the transcript. A vendor says the model has been updated. A consultant turns the incident into a slide titled “AI risk is real.” Everyone nods gravely. Very mature. Very enterprise. The harder question is less theatrical: can the same vulnerability be tested again, under controlled conditions, with visible logs, a consistent evaluator, repeatable statistics, and enough human inspection to make the result defensible? ...

A benchmark score is easy to quote. It is harder to know what broke. In Jailbreak Mimicry: Automated Discovery of Narrative-Based Jailbreaks for Large Language Models, Pavlos Ntais reports an 81.0% attack success rate against GPT-OSS-20B on a held-out 200-item test set.1 That number is attention-grabbing. It is also not the main lesson. ...

Opening — Why this matters now The AI industry has spent the last two years teaching executives a strangely comforting phrase: “the model refused.” That phrase is now dangerously inadequate. A refusal is not a security architecture. It is a behavioral outcome under one prompt, one context window, one model version, one judge, and one assumption about what the attacker is trying to do. Change any of those variables and the safety story can change. Sometimes gently. Sometimes like a glass door discovering what gravity does. ...

A refusal is easy to recognize. The model says it cannot help. The sentence sounds polite. The compliance team relaxes for three seconds. Everyone moves on. That is the comfortable version of AI safety: refusal as an observable behavior. The uncomfortable version is that refusal may be only the visible end of a much narrower internal computation. If that computation can be found, isolated, and steered, then the model’s “sorry, I can’t assist with that” is not a moral boundary. It is a circuit behavior. Very reassuring, in the same way a locked glass door is reassuring before someone points out the hinge. ...

A support ticket does not usually arrive as a clean moral philosophy exercise. It arrives as a complaint marked urgent. Then the customer adds that a manager already approved something questionable. Then a sales team wants the answer phrased in a way that protects revenue. Then the user says there is no time to escalate. Five turns later, the AI assistant is no longer answering the original question. It is swimming inside pressure, ambiguity, and incentives. ...

Email is a boring place to start an AI security article. That is exactly why it is useful. A modern enterprise agent is not merely answering questions about email. It can search messages, summarize attachments, update calendars, create rules, contact colleagues, write to Slack, edit files, and remember what it learned for next time. In demo videos, this looks like productivity. In security reviews, it looks like a small software system that accepts natural language as both instruction and evidence. Wonderful. We have reinvented workflow automation, except now the workflow engine reads every suspicious paragraph with a helpful attitude. ...

Email is a boring interface. That is exactly why it is dangerous. A user asks an AI agent to summarize a message, update a record, book a trip, or search a workspace. The agent reads some external content, decides which tool to call, fills in the parameters, and continues the user’s task. Somewhere inside that external content sits a hidden instruction saying, in effect: “Before doing the user’s task, do mine.” ...

Jailbreaks are not polite enough to wait their turn. That is the awkward weakness in many safety-training pipelines. A model is attacked, patched, tested, and released. Then another attack appears, usually crafted with more creativity than the previous defense assumed. The safety team patches again. The benchmark improves. The real attack surface moves. Everyone calls this iteration, because “organized whack-a-mole with GPUs” sounds less respectable. ...

A user sounds distressed. They ask a factual question. The assistant responds warmly, offers supportive resources, and then supplies the requested information in crisp, well-organized detail. That is the failure pattern. Not because the model was rude. Not because it ignored crisis language. Not because it forgot to add a disclaimer. The problem is more uncomfortable: the model noticed enough to sound caring, but not enough to change what it was willing to provide. ...

Security teams already know the scene. A scanner produces a long list of suspicious services, outdated servers, odd access rules, and “maybe this is bad” findings. Then the real work begins: deciding which lead matters, proving impact without breaking production, writing a report someone can act on, and not getting distracted by every shiny port that waves from the network. ...