Bias on Demand: When Synthetic Data Exposes the Moral Logic of AI Fairness

Bias on Demand: When Synthetic Data Exposes the Moral Logic of AI Fairness

In the field of machine learning, fairness is often treated as a technical constraint — a line of code to be added, a metric to be optimized. But behind every fairness metric lies a moral stance: what should be equalized, for whom, and at what cost? The paper “Bias on Demand: A Modelling Framework that Generates Synthetic Data with Bias” (Baumann et al., FAccT 2023) breaks this technical illusion by offering a framework that can manufacture bias in data — deliberately, transparently, and with philosophical intent.

Rather than fixing bias, the authors turn the problem inside out. Their synthetic data generator can inject specific forms of bias — historical, measurement, representation, or omitted-variable — each formalized mathematically. This allows researchers to observe, under controlled conditions, how different biases distort fairness metrics and interact with mitigation techniques. In effect, the paper builds a laboratory for social harm.

From Hidden Bias to Controlled Experimentation

Traditional fairness research relies on real-world data — messy, incomplete, and entangled with countless social dynamics. The authors recognize that this makes it nearly impossible to isolate the causal role of a specific bias. Their innovation is to decompose bias into modular, reproducible components. For instance:

This design allows the team to simulate fairness trade-offs systematically, testing which mitigation techniques actually work. The experiments reveal that post-processing methods can effectively mitigate many biases — except measurement bias, which remains stubbornly resistant. Even more intriguingly, they find that removing sensitive attributes (so-called “fairness through unawareness”) can paradoxically increase unfairness.

Ethics in the Equation

Baumann and colleagues go further than mere diagnostics. They connect these bias types to moral worldviews — egalitarian, prioritarian, sufficientarian — showing that each fairness metric encodes a particular ethical philosophy. For example, demographic parity assumes equal outcomes are always desirable, but in some contexts, this “leveling down” can harm both majority and minority groups. By linking mathematical formalisms to moral logics, the paper bridges the gap between algorithmic design and political philosophy.

This moral explicitness matters. As the authors note, fairness debates often degenerate into metric wars — PPV parity vs. FOR parity vs. equalized odds — without recognizing that these metrics embody conflicting values about justice. Their framework enables what we might call ethical debugging: tracing how fairness failures stem not just from bad data, but from implicit moral assumptions.

Why Synthetic Data is a Moral Mirror

Synthetic data is not merely a privacy tool or a statistical convenience. In this context, it becomes a mirror for the moral structure of AI systems. By generating biased worlds on demand, researchers can stress-test fairness claims under transparent, repeatable conditions. This redefines synthetic data as a philosophical instrument — one that can reveal whether our fairness interventions actually produce justice, or just cleaner spreadsheets.

The implications are broad. Regulatory frameworks like the EU AI Act increasingly require explainability and bias audits. Tools like Baumann’s bias generator could make these audits more rigorous — turning ethical principles into testable hypotheses.

A Step Toward Equitable AI — and Honest Science

The most profound contribution of this work is epistemic: it reminds us that fairness is not an emergent property of code, but a choice about the kind of world we want AI to reproduce. By providing a toolkit to decompose and test these choices, the paper moves fairness research from moral rhetoric to scientific accountability.

In doing so, Bias on Demand does not promise fairness — it demands transparency. And perhaps that’s the fairer starting point.

Cognaptus: Automate the Present, Incubate the Future.