Local Fluency Is Not Local Fairness: IndoBias and the Indonesian Bias Problem
TL;DR for operators IndoBias is a useful paper because it attacks a lazy assumption: that a model becomes fairer in a country once it becomes more fluent in that country’s language. Charming idea. Unfortunately, culture is not a plugin. The paper introduces a two-track benchmark for bias in Indonesian and three local languages: Javanese, Sundanese, and Makasar. The first track, IndoBias-Pairs, uses 544 contrastive stereotype pairs per language to test whether a model assigns higher likelihood to prototypical statements than to counter-stereotypical ones. The second track, IndoBias-QA, uses generation-based prompts across 336 demographic groups to examine stereotype polarity at broader coverage, including groups that may not have widely agreed stereotype pairs. ...