Black Box Is Too Blunt: What AI Interfaces Reveal to Attackers
TL;DR for operators What exactly should a deployed AI system reveal to users, vendors, insiders, or connected applications? Hiding model architecture and weights does not make every interface equally opaque. Mahbub and colleagues separate access into six operational categories—None, Metadata, Decision-Only, Score/Rank, Embedding, and White-Box—because each exposes a different signal to an adversary.1 A binary decision permits probing, while numerical confidence or similarity values provide directional feedback; internal feature representations expose still richer information. The paper’s synthesis suggests that richer signals generally reduce attacker uncertainty and query burden while enabling additional risks such as model extraction and biometric template inversion. ...