The Leak That Barely Moved the Score
TL;DR for operators A medical-image classifier can retain excellent benchmark accuracy even when its evaluation data contain serious integrity defects. In one public brain-tumor MRI benchmark, 28.75% of test images had near-duplicates in training. Removing them changed balanced accuracy by only -0.009 on average across nine architectures, meaning the cleaned scores were slightly higher. ...