Data-Poisoning

TL;DR for operators A pre-training contamination test can produce a clean-looking power law without producing a reusable contamination rule. In controlled OLMo-style runs, increasing the poison fraction approximately follows a power law in relative clean-validation perplexity degradation, but the fitted relationship changes over training and with model size. Halder, Dey, and Pehlevan’s A Solvable Theory of Pre-training Data Poisoning: Regime-Dependent Scaling Exponents1 explains why such movement may be structural rather than measurement noise. ...

TL;DR for operators Publishing images that remain useful to people while making them poor training material for unauthorized models creates an awkward engineering constraint: stronger interference often comes with more visible image degradation. The paper Leveraging Imperfect Restoration for Data Availability Attack1 shows that those two objectives need not move together. Its proposed method deliberately restores some of the visual damage created by an existing convolution-based poisoning technique while leaving enough class-specific structure to continue disrupting learning. ...

TL;DR for operators A retrieval-security team can reasonably start by looking for poisoned documents that repeat a user’s question or arrive as an unusually tight cluster of near-duplicate vectors. CamoDocs shows why neither signal should be treated as a durable property of poisoning. In matched-budget experiments, a simple query detector keeps three earlier attacks between 5.4% and 7.6% attack success, while CamoDocs reaches 75.2%. Put the target query back into CamoDocs, and success falls to 7.8%. The paper’s ablations then isolate the harder problem: deliberately spreading poisoned-document embeddings raises attack success against the clustering-based TrustRAG defense from 11.5% to 28.7%. ...

TL;DR for operators A platform can evaluate a recommender on clean historical data, observe only a small performance gap between groups, and reasonably approve it for retraining. That approval does not show how the same training process will respond when coordinated fake accounts deliberately shape the next batch of user interactions. In the reported experiments, fake profiles widened subgroup disparities even when the target recommender used fairness-aware training. Across the tested models, the paper’s SRLFA method generally produced larger disparities than the adapted attack baselines, with the largest reported effects appearing on the fairness-aware Last.fm LightGCN target. ...

Customer support bots are not supposed to have enemies. They sit politely inside enterprise websites, read policy documents, retrieve relevant snippets, and answer questions with the soft confidence of a well-trained assistant. The selling point is simple: Retrieval-Augmented Generation, or RAG, should make large language models less likely to hallucinate because the answer is grounded in external evidence. ...