AI-Security

Email is a boring place to start an AI security article. That is exactly why it is useful. A modern enterprise agent is not merely answering questions about email. It can search messages, summarize attachments, update calendars, create rules, contact colleagues, write to Slack, edit files, and remember what it learned for next time. In demo videos, this looks like productivity. In security reviews, it looks like a small software system that accepts natural language as both instruction and evidence. Wonderful. We have reinvented workflow automation, except now the workflow engine reads every suspicious paragraph with a helpful attitude. ...

Email is a boring interface. That is exactly why it is dangerous. A user asks an AI agent to summarize a message, update a record, book a trip, or search a workspace. The agent reads some external content, decides which tool to call, fills in the parameters, and continues the user’s task. Somewhere inside that external content sits a hidden instruction saying, in effect: “Before doing the user’s task, do mine.” ...

Security teams do not need an AI that sounds decisive. They already have enough decisive systems. Some of them are called “legacy tools.” Some are called “urgent executive dashboards.” A few are called “we should probably reboot it.” What security operations need is more uncomfortable: an AI system that can propose useful response actions, explain why they might work, and then refuse to act when its own reasoning becomes unstable. That refusal matters. In an incident-response workflow, a hallucinated recommendation is not merely a bad paragraph. It can isolate the wrong host, patch a vulnerability that does not exist, wipe evidence too early, or generate a playbook that looks official while quietly wasting the first thirty minutes of response time. ...

Image security has an awkward habit of sounding theoretical until the image is inside a business workflow. A product team adds an image-upload feature. A compliance team uses multimodal models to inspect screenshots. A support bot reads photos from customers. A research assistant summarizes figures from PDFs. Everyone understands that the model may occasionally misread an image. That is ordinary error. Annoying, but ordinary. ...

A chatbot should not be the only employee in the company responsible for keeping secrets. That sounds obvious until we look at how many enterprise RAG systems are designed. A user asks a question. The system retrieves internal documents. The documents are placed into the model context. A policy instruction is added somewhere above the user prompt: do not reveal sensitive information. Then everyone hopes the model behaves. ...

Diagram reviews are where many security problems first become visible. Not in the production logs. Not in the postmortem. Not after a user discovers that a tool-calling agent has confidently pushed private data into the wrong API. The humble architecture diagram is supposed to be the place where adults in the room ask: what can go wrong here? ...

Accuracy is a comforting number. That is precisely the problem. A neural network can score well on a test set and still be operationally suspicious. The labels may be corrupted. The input may be degraded. A small patch may have quietly hijacked part of the model’s learned behavior. The model may be confident, calibrated enough for a dashboard, and still untrustworthy in the one place where the business actually needs it to behave. ...

A fraud model does not only learn from transactions. It learns from sequence. Who interacted with whom. When. How often. After what previous event. Before which next event. In temporal graph systems, the order is not metadata. It is the thing being modelled. That is why LoReTTA is an uncomfortable paper.1 It does not argue that Temporal Graph Neural Networks can be broken only by a powerful adversary with model access, expensive surrogate training, and a theatrical pile of fake edges. It argues something more operationally annoying: a continuous-time graph can be poisoned by removing influential interactions and replacing them with plausible ones. The resulting history still looks enough like history. The model quietly learns the wrong temporal structure. Very civilised, as crimes go. ...

Safety teams like switches. They are easy to name, easy to diagram, and easy to pretend are under control. For language models, “refusal” has often been treated with roughly that mental model. A harmful prompt enters. Somewhere inside the model, a refusal feature lights up. The model says no. If researchers can identify the feature, they can study it, steer it, strengthen it, or—less comfortably—remove it. ...

TL;DR for operators Tools are where agent security stops being philosophical. Once an AI agent can read files, call APIs, inspect environment variables, launch commands, or connect to a database, the business question is no longer “is the model aligned?” It is “what exactly can this process touch when it is confused, manipulated, or supplied with a malicious tool?” ...