Who’s Really in Charge? Epistemic Control After the Age of the Black Box

Opening — Why this matters now

Machine learning has become science’s most productive employee—and its most awkward colleague. It delivers predictions at superhuman scale, spots patterns no graduate student could ever see, and does so without asking for coffee breaks or tenure. But as ML systems increasingly mediate discovery, a more uncomfortable question has resurfaced: who is actually in control of scientific knowledge production?

The worry is not new, but it has sharpened. When models become opaque, adaptive, and partially autonomous, scientists risk losing not just interpretability, but something deeper—epistemic control. If we cannot tell why a system produces a result, or which standards it is answering to, are we still doing science, or merely supervising an alien epistemic process?

Emanuele Ratti’s paper intervenes precisely here, offering a rare move in the debate: neither techno-optimism nor black-box fatalism, but a careful reconstruction of what control in ML-based science actually means—and how it subtly mutates.

Background — From human-in-the-loop to epistemic control

Much of the public debate about AI revolves around meaningful human control, a concept forged in discussions about autonomous weapons, public-sector algorithms, and credit scoring. There, the concern is moral and legal responsibility.

Ratti transposes this framework into science and sharpens it. In scientific contexts, control is not about blame—it is about knowledge. He calls this epistemic control, defined through two conditions:

• Tracking: scientific tools must be responsive to the epistemic and methodological standards of a discipline.
• Tracing: scientists must be able to reconstruct, using discipline-specific concepts, how a result was produced.

Epistemic control is not binary. It exists on a continuum. A system can be well-tracked but poorly traced, or vice versa. And crucially, control does not guarantee correctness—it merely enables scrutiny.

This framework allows Ratti to translate a long-standing critique by philosopher Paul Humphreys into operational terms. Humphreys argues that ML introduces a new, non-human epistemic perspective—one grounded in opaque internal representations that resist translation into human scientific concepts. If that translation fails, both tracking and tracing collapse.

Analysis — Why the pessimism is overstated

Ratti agrees that ML systems—especially deep neural networks—are representationally opaque. Their learned embeddings are implicit, distributed, and often uninterpretable within existing scientific ontologies. If your gold standard is mechanistic explanation, this is indeed a problem.

But here comes the first corrective move: opacity only threatens epistemic control relative to certain scientific goals.

In biology, for example, ML models often fail spectacularly as mechanistic explanations. You cannot extract a clean causal story from a high-dimensional classifier. But explanation is not the only epistemic aim. Classification, stratification, and prediction are equally legitimate scientific objectives—and for these, tracking conditions can still be satisfied through external validation, clinical coherence, and robustness checks.

In short: ML does not expel humans from science. It just refuses to play by explanatory rules it was never designed for.

The second correction is more subtle—and more interesting. Humphreys implicitly treats ML’s internal representations as the entire epistemic perspective of ML-based science. Ratti points out that this is a category error.

Most of an ML system is not learned—it is designed.

Problem formulation, data curation, performance metrics, training regimes, deployment thresholds: all of these are shaped by human cognitive values such as accuracy, robustness, parsimony, generalization, and fairness. These values are not merely present; they are continuously specified, traded off, and renegotiated.

Ratti distinguishes two moments:

1. Value specification – deciding what counts as accuracy, efficiency, or representativeness.
2. Value choice – deciding which values dominate when trade-offs are unavoidable.

This deliberative process is irreducibly human. It anchors tracking and tracing even when internal representations remain opaque. The black box, it turns out, is wrapped in a very human envelope.

Findings — Control shifts, it doesn’t disappear

Ratti’s most original contribution arrives late in the paper, and it is where the real tension lies. Even if ML systems are shaped by human values, they push back.

Machine learning brings its own normativity—not in the sense of moral agency, but in the sense of constraints. To function properly, ML-based science requires scientists to accept certain implicit norms: favoring prediction over explanation, scalability over interpretability, statistical performance over causal intelligibility.

This creates a two-way dynamic:

In domains like genomics, the consequence is clear. As ML tools become central, disciplines recalibrate what counts as good science. Mechanistic understanding recedes; predictive success advances. This is not coercion—but it is not neutral either.

Epistemic control survives, but it becomes partial, negotiated, and historically contingent.

Implications — The quiet politics of methodology

The real risk of ML-based science is not that humans are replaced by machines. It is that methodological shifts occur without explicit reflection.

When scientific communities adopt ML tools, they also—often implicitly—adopt the values those tools privilege. Over time, this can narrow the space of legitimate questions, marginalize certain explanatory practices, and redefine epistemic success.

This is not unique to ML. Every major methodology has done this. But ML accelerates the process and hides it behind technical sophistication.

For business leaders, policymakers, and research managers, the lesson is simple but uncomfortable: investing in ML is also investing in a particular vision of knowledge. If you do not articulate which epistemic values you are willing to trade away, the system will decide for you.

Conclusion — Control, but with conditions

Ratti’s argument lands in a carefully balanced place. Machine learning does not render science post-human. Nor does it leave epistemic control untouched. Instead, it reconfigures control, redistributing it across design choices, evaluation metrics, and shifting disciplinary aims.

The challenge ahead is not to demand full transparency where none is possible—but to remain vigilant about which standards we allow machines to enforce by default.

Black boxes do not rule science. But they do negotiate the terms.

Cognaptus: Automate the Present, Incubate the Future.