LLM Security

TL;DR for operators Encrypted LLM connections can conceal prompts and responses while exposing recognizable patterns of AI activity. In a controlled benchmark, classifiers using only encrypted response metadata identify broad prompt categories with 67.37% mean accuracy, against a 16.67% chance baseline. Traffic from collaborative agents also reveals task categories, including when only part of the team’s communication is visible. ...

TL;DR for operators A retrieved document does not need to look malicious to corrupt a generative-search answer. In Counter-GEO-Bench, carefully constructed documents remain fluent and topically relevant while inserting a targeted false claim; in the undefended pipeline, 55.7% of these attacks move the generated answer toward that claim across three tested models.1 ...

TL;DR for operators A model can pass an ordinary unlearning evaluation while still yielding supposedly forgotten information when the request is reformulated strategically. Gupta and colleagues demonstrate this gap in a controlled benchmark of LLM unlearning: on the 1% forget split, four fine-tuning-based methods produced average adversarial recovery rates between 72.8% and 84.3%, compared with 87.5% for the unprotected model.1 ...

TL;DR for operators A retrieval-security team can reasonably start by looking for poisoned documents that repeat a user’s question or arrive as an unusually tight cluster of near-duplicate vectors. CamoDocs shows why neither signal should be treated as a durable property of poisoning. In matched-budget experiments, a simple query detector keeps three earlier attacks between 5.4% and 7.6% attack success, while CamoDocs reaches 75.2%. Put the target query back into CamoDocs, and success falls to 7.8%. The paper’s ablations then isolate the harder problem: deliberately spreading poisoned-document embeddings raises attack success against the clustering-based TrustRAG defense from 11.5% to 28.7%. ...

TL;DR for operators Prompt-injection defense should not be procured as a leaderboard winner. A systematic review of 88 studies finds defenses distributed across several parts of the LLM system, from training and prompt handling to document boundaries, tool execution, output filtering, and continuous testing.1 Fifty-six of those 88 approaches, or 63.63%, are model-agnostic: they can operate around different models without changing model weights or architecture. That matters for teams building on proprietary APIs. ...

TL;DR for operators A production AI request may pass through preprocessing, a safety guardrail, and a generator, with each stage intended to reduce risk. Cascade1 shows why evaluating those defenses separately can miss an important failure mode: an exploit at one layer can remove a condition that another defense depends on. In the paper’s guardrail experiment, random attention perturbation evades the guardrail on 94% of evaluated malicious prompts, versus 82% for targeted token bitflips and 72% for targeted attention bitflips. That 94% figure is not an end-to-end compromise rate. After combining guardrail evasion with a reported 82% generator-jailbreak rate and externally sourced hardware bitflip probabilities, the corresponding calculated full-chain attack-success rate is 0.750 or 0.765. ...

TL;DR for operators An embodied agent can receive the correct user instruction and still plan toward the wrong objective if the internal description of its environment has been manipulated. Liu et al. test this failure mode by altering planner-visible state semantics rather than changing the instruction, model, planner, executor, or environment itself.1 ...

TL;DR for operators The current enterprise LLM mistake is to treat safety as a chat-interface problem. Put a filter in front, add a stern system prompt, run a cheerful demo, and hope the model has suddenly acquired a moral philosophy and a compliance department. Charming. Also insufficient. Two recent papers make a stronger and more operationally useful point. The first evaluates open-source LLMs under prompt-injection and jailbreak attacks, then compares lightweight inference-time defenses such as input filtering, system-prompt hardening, vector detection, voting, and self-examination.1 The second studies whether fine-tuned language models memorize sensitive personally identifiable information that appears only in inputs, not in the desired outputs, and benchmarks mitigation methods such as differential privacy, UnDial, regularization, and DPO.2 ...

TL;DR for operators The paper introduces GAS-Leak-LLM, a black-box method that uses a genetic algorithm to evolve adversarial suffixes: small text sequences appended to harmful prompts to increase the chance that a model produces unsafe content.1 The important part is not that another jailbreak exists. We have enough of those. The important part is that jailbreak discovery is framed as a repeatable optimization loop using only model queries. ...

Security teams like to search for suspicious strings. That habit is understandable. Strings are visible. They can be logged, filtered, matched, scored, and proudly displayed in dashboards. A bad suffix at the end of a prompt looks like a bad suffix at the end of a prompt. Convenient. Almost too convenient. The problem is that prompts are not flat text boxes. They are transformed into token sequences, wrapped in chat templates, and passed through attention layers that do not treat every position equally. Some positions receive more influence over the model’s next-token behavior than others. Put adversarial tokens there, and the same amount of “badness” can travel farther. ...