When Consensus is Just Noise: The Lottery Inside Collective AI

Opening — Why this matters now

Multi-agent AI systems are quietly becoming the operating system of modern decision-making. From financial trading bots to policy simulations and automated research pipelines, we are increasingly asking groups of models to produce answers rather than relying on a single one.

And when they agree, we tend to relax.

Agreement feels like validation. Consensus feels like intelligence.

This paper—fileciteturn0file0—suggests something less comforting: sometimes consensus is not intelligence at all. It is a statistical accident that got amplified.

Not wisdom. Not bias. Just… drift.

Background — Context and prior art

The idea that groups can be smarter than individuals is not new. Collective intelligence has long been studied in economics, sociology, and statistical physics. Classic models like the naming game show how shared conventions emerge from repeated interactions.

What changes with LLMs is scale and opacity.

Unlike human groups:

• AI agents do not have stable beliefs.
• They do not “remember” in a persistent sense.
• Their outputs are probabilistic samples, not deterministic statements.

Recent work has shown that even when all agents start neutral—with no preference among options—they still converge to a shared answer. That alone raises an uncomfortable question:

If no one prefers anything, why does everyone end up agreeing on something?

This paper answers: because they accidentally convince each other.

Analysis — What the paper actually does

1. The hidden mechanism: Mutual in-context learning

In standard LLM usage, a model learns from a fixed prompt distribution. In multi-agent systems, something stranger happens:

• Each agent generates an output
• Other agents treat that output as evidence
• That evidence updates their internal state

This creates a feedback loop where the system becomes its own training data.

The authors call this mutual in-context learning.

And it has a peculiar property: early randomness can snowball into certainty.

2. Memetic drift: When randomness becomes reality

Borrowing from evolutionary biology, the paper introduces the concept of memetic drift:

• No option is objectively better
• Small random fluctuations occur
• These fluctuations get reinforced through interaction
• Eventually, one option dominates

In other words, the “winner” is often just the first lucky sample.

This is not bias. It is not reasoning.

It is amplified noise.

3. The QSG model: A minimal but dangerous abstraction

To formalize this, the paper introduces Quantized Simplex Gossip (QSG):

Each agent:

• Holds a probability distribution over possible answers
• Communicates samples (not full beliefs)
• Updates based on what others say

The key update rule is deceptively simple:

$$ x_L \leftarrow (1 - \alpha)x_L + \alpha y $$

Where:

• $\alpha$ = adaptation rate
• $y$ = sampled message from another agent

That’s it.

No rewards. No ground truth. No optimization objective.

Yet the system still converges.

Which should make you slightly uneasy.

4. The real insight: Consensus is a scaling phenomenon

The paper’s core contribution is not the model—it’s the scaling laws.

They show that whether consensus is meaningful or random depends on four variables:

From this, a critical insight emerges:

Consensus is not binary. It is a regime.

There are two distinct worlds:

The transition between them is predictable.

And dangerously easy to misinterpret.

Findings — What actually happens (with structure)

Drift vs Selection Dynamics

Key Quantitative Patterns

These are not just theoretical.

The paper validates them using real LLM populations (GPT-4o, Claude Haiku), showing near-perfect alignment with the predicted scaling laws.

Which is both impressive—and slightly alarming.

Implications — What this means in practice

1. Agreement is not evidence

If your multi-agent system reaches consensus, that does not mean:

• The answer is correct
• The agents reasoned effectively
• The system aggregated information properly

It may simply mean:

The system got lucky early—and doubled down.

2. Small systems are especially dangerous

In low-agent or low-communication setups:

• Outcomes are highly path-dependent
• Different runs produce different “truths”
• Reproducibility collapses

This has direct implications for:

• AI-driven trading strategies
• Automated policy simulations
• Multi-agent decision pipelines

3. Faster is not better

Higher adaptation rates ($\alpha$):

• Speed up convergence
• But amplify noise relative to signal

In business terms:

The system becomes confidently wrong, faster.

4. Bias doesn’t need to be strong to dominate

Even tiny asymmetries (prompt wording, ordering, memory effects):

• Get amplified in large systems
• Eventually determine the outcome

Which means:

You are always encoding a bias—even when you think you are not.

5. Governance must shift from outputs to dynamics

Traditional evaluation asks:

• Is the answer correct?

This paper suggests a better question:

• Why did the system converge to this answer?

That requires monitoring:

• Interaction structure
• Information flow
• Variance and drift metrics

In other words: process, not just outcome.

Conclusion — The uncomfortable truth about “collective intelligence”

We like to believe that more agents mean more intelligence.

This paper suggests a more precise formulation:

More agents mean more amplification.

Sometimes that amplifies signal.

Sometimes it amplifies noise.

And without careful design, you won’t know which one you’re getting.

Consensus, then, is not a guarantee of intelligence.

It is a phase of a system.

And occasionally, it is just a very convincing lottery.

Cognaptus: Automate the Present, Incubate the Future.