Ready Player None: Why AI Still Can’t Beat the Human Game Multiverse

Games are not supposed to be frightening.

A commuter plays them between meetings. A child learns one in thirty seconds. A bored adult opens a mobile puzzle, fails once, notices the trick, and improves. No dissertation. No onboarding deck. No “agentic workflow architecture.” Just look, act, remember, adjust.

That is precisely why the new AI GAMESTORE paper is awkward for the current AI narrative.¹ It does not ask whether frontier models can solve another static exam, write another function, or produce another polished paragraph about strategic transformation. They can do all of that, often impressively. The paper asks something more ordinary and therefore more damaging: can a model learn unfamiliar human-designed games under roughly human-like gameplay constraints?

The answer, in this first benchmark, is mostly no.

Across 100 short games adapted from popular digital-game concepts, the strongest evaluated vision-language models achieved less than 10% of median-human performance on a human-relative score scale. The best model in the main figure, GPT-5.2, reached 8.5 when the human median was set to 100. The models also took far longer in wall-clock time: humans played each game for 120 seconds, while models commonly required more than 1,200 seconds to complete the same 120-step gameplay process.

That headline is memorable. It is also the least interesting part of the paper.

The more useful contribution is the measurement design: a living, human-relative, capability-annotated benchmark for interactive learning. AI GAMESTORE matters because it shows how evaluation can move from “Can the model answer a known class of questions?” to “Can the model enter a new environment, infer the rules, keep state, plan actions, and improve at the pace of a human learner?”

For business readers, that distinction is not academic decoration. It is the gap between a demo and a deployable agent.

The mechanism: evaluate learning inside human-made task worlds

The paper begins with a simple reframing. Instead of defining general intelligence through abstract task sets or static exams, the authors propose the “Multiverse of Human Games”: the open-ended distribution of games humans could design, enjoy, and teach to one another.

This is not just a cute metaphor. Games are compressed environments. They package perception, rules, incentives, memory, timing, planning, physical reasoning, social reasoning, and world-model discovery into bounded systems with measurable outcomes. A game is a miniature world where the agent must do more than recognize a pattern. It must act.

That makes games different from most benchmark items. A static question can reward stored knowledge, pattern completion, or benchmark-specific optimization. A game pressures the system over time. The agent must perceive the screen, choose an action, observe the result, update its internal model, and try again. Failure compounds. So does confusion.

The paper’s mechanism has three important pieces:

That third part is the quiet center of the paper. A model score by itself tells us that something failed. A capability profile starts to tell us what failed.

This is the point many business discussions about AI agents still miss. “The agent failed” is not an explanation. Did it fail because the interface was bad? Because it forgot prior state? Because it could not infer a hidden rule? Because it saw the screen but could not plan a sequence? Because the action loop was too slow? Without separating these failure modes, evaluation becomes a ritual. Everyone nods solemnly at a benchmark number and then goes back to prompt engineering, because apparently that is what civilization has chosen.

AI GAMESTORE is trying to make the failure more legible.

AI GAMESTORE turns an impossible multiverse into a controlled factory

Evaluating all possible human games is impossible. The paper does not pretend otherwise. Instead, it builds a practical proxy.

The pipeline starts from real digital-game marketplaces. For the first version, the authors sourced Apple App Store top games across five categories in 15 countries, producing 7,500 App Store candidates, and also included top indie games from Steam. The candidates were filtered for popularity, ratings, diversity, and suitability: playable within minutes, expressible in p5.js, measurable by score, and not dependent on specialized prior knowledge like poker.

Then comes the important engineering compromise. The authors do not run commercial games directly. That would bring copyright issues, interface heterogeneity, data contamination, and real-time latency problems. Instead, they use LLMs to generate standardized p5.js versions inspired by those game concepts.

The generation process is not fully automated, and that is a strength rather than a weakness. The paper uses a human-in-the-loop refinement cycle. A model generates an initial game. Automated tests catch mechanical bugs. Human players then play the game and provide feedback until it is playable and engaging enough. On average, the paper reports 4.7 refinement steps per game, with each refinement step taking about two minutes. The full game generation and refinement process takes roughly 30 minutes on average.

That gives AI GAMESTORE its main practical shape:

This is why the benchmark is more than “models play games.” It is a controlled environment factory. The paper’s durable idea is that evaluation tasks can be generated, standardized, privately held, refreshed, and annotated. Static benchmarks become targets. Living benchmarks become moving terrain.

That matters because most real AI deployments are moving terrain.

A customer-support process changes. A compliance rule is interpreted differently by a regulator. A supplier portal moves a button. A user behaves in a way the designer did not expect. The system must adapt without being retrained on a perfectly frozen exam. Games are not the business environment, but they are a useful miniature of the same problem: unfamiliar rules, sequential action, partial feedback, and the need to improve before the clock runs out.

The benchmark measures rapid learning, not grandmaster expertise

The experiment is easy to misunderstand.

This is not a claim that humans are better than specialized game AIs at all games. That would be silly; we settled chess and Go long ago, and the machines were not subtle about it. The question here is not whether a specialized system can master one known environment after heavy training. The question is whether a general-purpose model can rapidly learn many unfamiliar human-designed games with limited experience.

The human side used 106 participants recruited through Prolific. Each participant played 10 games, with 120 seconds per game. Scores were collected during play, and participants also rated games for fun and challenge.

The model side evaluated seven frontier vision-language models:

The scoring method normalizes each model’s raw score against the median human score for that same game:

Then the authors report geometric means across games, which is appropriate because game scores are heterogeneous and skewed. A model that performs decently on a few games but collapses on many should not look healthy just because one raw scoring system was generous.

This is a measurement of breadth under pressure. The models receive screenshots, game descriptions, recent actions, and a scratchpad. Every gameplay second, the game pauses and the model produces five action lists, each corresponding to 0.2 seconds of play. That harness is generous in one sense: it pauses the game so the model can think. It is restrictive in another: the model interacts through a discrete action interface rather than natural human motor control.

The paper is careful enough not to treat this harness as final. It explicitly encourages alternative harnesses and agent architectures. That matters because the benchmark tests models under one interaction protocol, not every possible future embodied agent design.

Still, the result is not flattering. Even under a pause-and-query setup designed around current model latency, frontier VLMs are far below ordinary human players.

The score is bad; the distribution is worse

An average score below 10% is already a problem. But the distribution of failures is more informative.

The paper reports a bimodal pattern. On roughly two-thirds of the games, models make some progress, although usually at only 10–30% of median-human performance. On roughly 30–40% of games, they fail to make meaningful progress at all, often scoring below 1% of the median human.

That is not smooth degradation. It is brittleness.

A robust learner should often be mediocre before it becomes good. These models often move from “somewhat functional” to “basically lost.” The interesting question is what separates those two regimes.

The paper’s answer is not simply “harder games.” It is the type and combination of cognitive demands.

AI GAMESTORE annotates each game across seven capability categories:

The models do not fail equally across this map. The paper finds especially serious bottlenecks in memory, planning, and world-model learning. Performance also falls as games demand more distinct capabilities at once. A model may survive a simple visual-processing task. Ask it to combine visual parsing, remembered state, inferred mechanics, and multi-step planning, and the system starts to look less like a general agent and more like a very eloquent tourist holding the controller upside down.

This distinction is crucial. The paper is not merely saying “models are bad at games.” It is saying that today’s models struggle when competence must be integrated over time.

That is exactly the part of intelligence that enterprise automation quietly depends on.

The reaction-time excuse does not survive the appendix

A convenient objection is available: perhaps the models fail because the harness is too slow for real-time games. That would be a comforting explanation. It would move the problem from cognition to latency. Latency is an engineering problem; cognition is less polite.

The authors test this objection by looking at games with low spatial-temporal coordination demand, meaning games that do not require fast reaction time. These include puzzle-like and turn-based environments where timing is less central. The result does not meaningfully rescue the models. The paper reports that top-model performance remains broadly similar on these low-reaction-time games.

This is a robustness check, not a second thesis. Its purpose is narrow but important: it weakens the claim that poor model performance is mostly an artifact of real-time motor speed. It does not prove the harness is perfect. It does not prove every model architecture would fail. But it does make one lazy explanation less available.

The appendix also compares median and geometric-mean performance. This supports the same broad picture from a different aggregation lens. Again, the point is not that one number is sacred. The point is that the performance gap is not a fragile artifact of a single scoring summary.

A useful way to read the evidence is:

This is how the paper should be interpreted: as a prototype for capability-oriented evaluation, not as a final universal scoreboard.

The business lesson is diagnostic sandboxes, not office arcade machines

The obvious business reading would be: “AI cannot play casual games, therefore AI agents are not ready.” That is too blunt.

The better reading is: many business AI systems are being evaluated at the wrong level of abstraction. They are tested on outputs, not learning dynamics. They are tested on completed tasks, not stateful adaptation. They are tested on demos, not capability stress.

AI GAMESTORE suggests a more useful pattern for enterprise evaluation:

1. Convert representative workflows into controlled interactive environments.
2. Give humans and AI systems comparable task budgets.
3. Measure not only completion, but progress over time.
4. Annotate tasks by cognitive or operational demand.
5. Diagnose failure by capability, not only by final score.
6. Refresh tasks continually to reduce overfitting.

That is the business relevance pathway.

For example, consider a procurement agent. A static benchmark might ask whether it can summarize vendor terms. A more useful benchmark would place it in a simulated procurement portal where vendor rules change, partial information appears across screens, previous decisions affect later options, and the agent must choose actions under a time or cost budget.

The resulting question changes:

That shift matters for ROI. Many AI projects fail not because the model cannot produce useful language, but because the system cannot maintain enough task state, infer enough process structure, or recover from enough small surprises. The expensive failure is not always a dramatic hallucination. Sometimes it is a quiet operational stall.

Games expose stalls.

In business workflows, a stall looks like an agent repeatedly clicking the wrong portal option, forgetting a constraint mentioned five screens ago, asking for human intervention too early, or confidently executing a plan that became invalid three steps ago. Replace the cartoon mouse and cheese with invoices, compliance checks, customer tickets, or logistics routing. The cognitive shape is less different than executives may prefer to admit.

Memory and world models are infrastructure problems, not prompt decorations

One of the paper’s more commercially important signals is that the weak points are not just surface reasoning errors. Memory, planning, and world-model learning are architectural stress points.

A scratchpad helps only if the system knows what to store, how to update it, when to retrieve it, and how to use it in action selection. A planning prompt helps only if the environment model is good enough to simulate consequences. A longer context window helps only if the agent can separate relevant state from noise.

This is why the paper’s result maps naturally to the next layer of AI product design.

For enterprise agents, the missing pieces often include:

This is where “just improve the prompt” starts to look like trying to repair a bridge with better adjectives. Prompts matter, but they do not replace task memory, environment modeling, action validation, and monitoring.

The paper does not solve these engineering problems. It provides a way to see them more clearly.

The benchmark is promising because it is imperfect in useful ways

AI GAMESTORE has real boundaries.

The current game suite is mostly short, casual, and simplified. The paper explicitly notes that it does not yet cover the full richness of human games, especially long-horizon environments, complex social reasoning, sophisticated multi-agent interaction, or extended narratives. Most current games are easy enough for humans to learn quickly. Some cognitive categories, such as deep social reasoning, are still weakly represented.

The model harness is also only one possible interface. Models interact through a pause-and-query mechanism with screenshots, a scratchpad, and action lists. Future agents with better real-time perception-action loops, specialized memory systems, or learned game-playing policies might perform differently.

The generated games are standardized p5.js adaptations, not the original commercial games. That is deliberate, but it means the benchmark is testing simplified variants, not the full sensory, social, and economic complexity of real games.

These limitations do not erase the signal. If anything, they make the initial result more uncomfortable. The benchmark is not yet throwing models into sprawling multiplayer worlds with deceptive humans, complex economies, and hour-long hidden objectives. It is testing short playable environments, many of which are deliberately casual. The models are already struggling.

The right conclusion is disciplined: AI GAMESTORE is not an AGI oracle. It is a proof-of-concept for a better evaluation direction.

What Cognaptus infers for business use

The paper directly shows that seven frontier VLMs perform far below humans on a first set of 100 AI GAMESTORE games, especially when tasks demand memory, planning, world-model learning, and capability integration.

Cognaptus infers a broader business lesson: enterprise AI evaluation should move toward dynamic, human-relative, capability-annotated sandboxes. Not because games are the same as business processes, but because both require stateful adaptation under feedback.

What remains uncertain is how much performance would improve under alternative harnesses, specialized agent architectures, longer training, richer memory systems, or task-specific tools. It is also uncertain how well game-derived capability profiles transfer to specific enterprise workflows. A procurement portal is not Flappy Bird. Thankfully.

But the evaluation pattern transfers well:

That is the practical value. The paper is not telling firms to benchmark their invoice agents on casual games. It is telling them to stop mistaking static performance for adaptive competence.

The multiverse is not impressed by polished text

AI has become very good at looking competent in static settings. It can summarize, classify, translate, draft, explain, and code. Those abilities are valuable. They are also not the same as learning to act in unfamiliar environments.

AI GAMESTORE is useful because it makes the gap visible in a form that is hard to talk around. The games are small. The human players are ordinary. The time budget is short. The models are strong. And still, the gap is large.

The most important lesson is not that frontier models scored 8.5 instead of 100. The important lesson is that interactive intelligence fails through mechanisms: forgotten state, weak planning, poor causal discovery, brittle integration, and slow action loops. Those mechanisms are exactly where the next generation of useful AI agents must improve.

For business leaders, the implication is clear enough: do not buy agentic capability based only on static benchmarks or beautiful demos. Ask how the system learns inside a process. Ask how it remembers. Ask how it recovers. Ask what happens when the rules are not fully written down.

The multiverse of human games is not impressed by fluent explanations.

Neither is the real world.

Cognaptus: Automate the Present, Incubate the Future.