AI Privacy

TL;DR for operators The current enterprise LLM mistake is to treat safety as a chat-interface problem. Put a filter in front, add a stern system prompt, run a cheerful demo, and hope the model has suddenly acquired a moral philosophy and a compliance department. Charming. Also insufficient. Two recent papers make a stronger and more operationally useful point. The first evaluates open-source LLMs under prompt-injection and jailbreak attacks, then compares lightweight inference-time defenses such as input filtering, system-prompt hardening, vector detection, voting, and self-examination.1 The second studies whether fine-tuned language models memorize sensitive personally identifiable information that appears only in inputs, not in the desired outputs, and benchmarks mitigation methods such as differential privacy, UnDial, regularization, and DPO.2 ...

AI risk reviews still tend to begin with comforting questions. Who is the responsible developer? What policy applies? What did the model output? Was the user allowed to ask that? Did the compliance team approve the deployment checklist? Useful questions, certainly. Also slightly late. Two recent arXiv papers point to a less convenient lesson: some AI risks are not merely produced by bad prompts, careless users, malicious deployment, or weak legal controls. They are produced by architecture. One paper shows this at the model-training layer, where Batch Normalization can amplify memorization of atypical samples and increase privacy leakage.1 The other shows it at the ecosystem layer, where decentralized AI can dissolve the very addressee that conventional governance assumes, forcing governance to move from policy instructions to protocol-level constraints.2 ...

TL;DR for operators Private examples are not harmless just because they sit inside a prompt rather than inside model weights. In-context learning lets teams adapt a general LLM by adding examples at inference time, which is convenient until those examples are medical notes, legal clauses, customer tickets, invoices, or internal decisions that should not be inferable from the model’s output. ...

TL;DR for operators Memorization audits usually start with the wrong question: “Which individual text snippets look memorized?” This paper suggests a better first diagnostic: group many snippets by how closely the model reproduces them, then measure the entropy of the token distribution inside each group.1 The result is an empirical pattern the authors call Entropy–Memorization Linearity. In plain English: when training examples are pooled by edit-distance score, their set-level entropy forms a strong linear relationship with how closely the model reproduces them. Since the paper’s “memorization score” is an edit distance, lower score means stronger verbatim reproduction; higher score means the generated continuation is farther from the ground truth. ...