A chatbot should not be the only employee in the company responsible for keeping secrets. That sounds obvious until we look at how many enterprise RAG systems are designed. A user asks a question. The system retrieves internal documents. The documents are placed into the model context. A policy instruction is added somewhere above the user prompt: do not reveal sensitive information. Then everyone hopes the model behaves. ...
A PDF is supposed to sit quietly. It may contain claims, equations, tables, and occasionally an appendix long enough to test a reviewer’s commitment to science. It is not supposed to negotiate with the system judging it. That assumption becomes unreliable once a document enters an LLM-based workflow. To the human reader, a sentence rendered in white text may be invisible. To a text-extraction pipeline, it can remain perfectly legible—and potentially indistinguishable from an instruction the model is expected to follow. ...
Scale Fail: How Downsampling Becomes an Adversarial Backdoor for VLMs Resize. It is one of those engineering verbs that sounds too boring to threaten anyone. A user uploads a screenshot, invoice, inspection photo, interface capture, medical form, or product image. The system resizes it. The model reads it. The workflow moves on. ...
Email is boring. That is its superpower. A message arrives. It looks like business sludge: compliance wording, project references, perhaps a polite request that nobody asked for. It contains no executable attachment, no obvious malware, no urgent invoice from a suspicious cousin. In a normal security review, it is background noise. EchoLeak makes that boring object more interesting. The paper examines CVE-2025-32711, a reported zero-click indirect prompt-injection exploit against Microsoft 365 Copilot, where a crafted external email could allegedly cause Copilot to leak internal information without the user clicking a malicious link.1 The central lesson is not that Copilot was uniquely careless, nor that prompt injection has suddenly become cyberpunk magic. The lesson is more uncomfortable: enterprise copilots are becoming data-flow infrastructure, and data-flow infrastructure fails when content, instructions, rendering, and network access are allowed to melt into one warm productivity soup. ...
Every executive agent demo eventually reaches the same awkward moment: the model stops being a chatbot and starts touching things. Files. APIs. Databases. Code runners. Email clients. Payment workflows. Production systems, because apparently we enjoy giving probabilistic text engines access to expensive buttons. The paper Architecting Resilient LLM Agents: A Guide to Secure Plan-then-Execute Implementations argues that the core safety problem is not merely that agents sometimes reason badly. The sharper problem is that many agent architectures let untrusted information change what the agent decides to do next.1 That is a control-flow problem. And control-flow problems are not solved by asking the model, very politely, to behave. ...
TL;DR for operators A bad agent incident rarely starts with one dramatic mistake. It usually forms as a chain. The system may be predisposed to fail because of training data, feedback, system prompts, or scaffolding. The environment may then trigger the failure through unclear tasks, insecure information, unavailable tools, excessive permissions, or malicious inputs. Finally, the agent may commit a visible cognitive error: it overlooks something, misunderstands a command, chooses the wrong goal, or executes an action badly. ...
TL;DR for operators A support agent reads a customer email. It checks a CRM record. It calls a refund API. It writes a note into long-term memory. It asks another agent to verify policy. Somewhere in that chain, a malicious instruction hides inside a message, document, issue tracker entry, retrieved snippet, schema, or tool response. The model does not need to become “evil”. It only needs to be helpful in the wrong direction. ...