Decontamination Is a Dial, Not a Delete Key
TL;DR for operators A benchmark score can become unreliable when test items, close paraphrases, or related material have entered training. Removing suspicious questions sounds straightforward, but any detector that misses contaminated items leaves score inflation behind, while filtering also changes the evaluation set. Chai, Zhe, and Sakuma propose DeconIEP,1 a white-box inference-time intervention that keeps the benchmark text and model weights fixed. Instead, it learns small, input-specific changes to the model’s embeddings so contaminated behavior moves closer to a comparatively less-contaminated reference model. ...
