TL;DR for operators

Giving an AI coding agent more opportunities to test and rewrite its software does not guarantee that it will produce a stronger program. In a benchmark spanning 12 competitive games, the strongest evaluated model–harness configuration reached rank 1 against six historical human-program ladders. None of the tested configurations reached rank 1 in the remaining six games. Tripling the interaction budget in four difficult cases produced two improvements, two unchanged results, and no additional first-place finish.

The more actionable finding concerns how agents acquire and use feedback. Detailed match replays outperformed simple win/non-win signals in all three games tested in a focused experiment. Opponent selection and test batch size also changed results, although their preferred settings varied by game.

For teams building autonomous software-optimization workflows, the research suggests investing in diagnostic execution traces, informative test selection, and versioned evaluation before simply increasing iteration allowances. The evidence supports this design direction within competitive games; it does not establish reliability in production business processes.

The agent keeps revising, but the ranking stops improving

Imagine an autonomous developer tasked with improving a resource-allocation program. It can inspect requirements, run tests, study failures, rewrite its decision rules, and repeat. Each unsuccessful test produces another opportunity to improve.

The difficulty emerges when the program keeps changing without becoming consistently better. A revision may correct one weakness while introducing another. More testing can reveal additional failures without identifying how to resolve them.

This is the problem examined by Kaisen Yang and colleagues in Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition.1

Their benchmark demonstrates that autonomous agents can develop highly competitive programs through repeated software revision. It also shows why continued access to feedback does not necessarily translate into continued improvement.

The distinction matters for any organization considering an AI system that autonomously improves executable decision rules. The resource-allocation question is not merely how many tests the system can afford, but whether those tests provide information the agent can turn into effective revisions.

AAArena measures sustained software development against human programs

The researchers introduce AAArena, a benchmark containing 12 adversarial games and 1,920 archived human-written programs from the Tsinghua University Agent Competition.

Unlike benchmarks that ask a language model to select actions directly, AAArena asks an AI developer to build a separate executable game-playing program. The developer receives rules, software development resources, and access to competitive feedback.

It can perform two kinds of evaluation:

  • Small matches: Targeted tests against selected opponents, providing detailed replay information for diagnosing behavior.
  • Full-pool evaluations: Broader assessments against frozen historical opponents, producing Elo ratings and leaderboard ranks.

The AI developer decides when to inspect, test, and revise within a default allowance of 128 small-match opponent tests and 16 full-pool evaluations.

This process formalizes Adversarial Heuristic Learning (AHL). The underlying AI model’s parameters remain fixed. What changes is the executable policy, its supporting software, and the accumulated development state.

The distinction is significant. A system can learn from experience in the operational sense of producing progressively better behavior without retraining its language model. The resulting improvements reside in the revised program.

AAArena scores the best officially evaluated policy retained during development, rather than necessarily the final revision. This prevents a late unsuccessful experiment from erasing the achievement of an earlier, stronger program.

Six first-place finishes, six unresolved games

The main benchmark evaluates seven model–harness configurations across all 12 games. Each main setting is repeated three times, with median retained-champion performance reported.

The strongest configuration, Opus5.5 operating through Claude Code, reaches rank 1 in six games. GPT6-sol and GLM-5.3 each reach rank 1 in four. The remaining configurations achieve between one and three first-place finishes.

Yet six games remain without a rank-1 result from any tested configuration.

The paper’s trajectory analysis makes the difference between progress and eventual success particularly clear. Across 12 GLM-5.3 game-development trajectories, four reach first place, five exhibit improvement interrupted by regressions, and three show persistent stagnation.

One detailed case illustrates what successful revision can accomplish. In Dorado, a resource-competition game, GLM-5.3 improves from rank 41 to rank 2. The development sequence includes changes addressing mine occupancy, resource allocation, hero upgrades, and conditional counterattacks.

These are substantive strategic revisions rather than cosmetic code changes. Nevertheless, the resulting program remains second in the historical ranking.

The researchers also observe that five of the six games with smaller rule descriptions have at least one first-place finish, compared with only one of the six games with larger descriptions. This association is consistent with rule understanding becoming a constraint on autonomous development. It does not establish causality: the rule-size measurements depend on the chosen representation and do not directly measure strategic difficulty.

Tripling the test budget does not remove every plateau

If agents can improve by inspecting failures, a natural proposal is to allow substantially more interactions.

The researchers test that possibility by continuing four difficult GLM-5.3 runs, extending their budgets from 128 small-match tests and 16 full-pool evaluations to 384 and 48, respectively. Existing workspaces and development histories are preserved.

Game Initial rank Extended-budget rank
Generals 17 17
Miracle 26 19
LOTA 22 22
AntWar2 9 6

Source: Section 4.3, Table 4. Four continued GLM-5.3 runs.

Two retained champions improve; two remain unchanged. None reaches first place.

This continuation experiment does not prove that these agents have reached a permanent capability ceiling. It establishes a narrower result: three times the nominal interaction allowance was insufficient to overcome the observed limitations in these four runs.

Additional tests help only when the agent can identify informative failures and translate them into successful strategic changes. If it repeatedly misunderstands a rule or implements an ineffective decision mechanism, another match may reproduce the same weakness.

The kind of feedback changes what the agent can learn

The paper’s focused experiments examine three design choices that determine the information available during development: which opponents to test, what feedback to provide, and how many opponents to include in each request.

The clearest result concerns feedback detail.

When GLM-5.3 receives detailed match replays rather than binary win/non-win signals, its retained programs perform better in all three tested games.

Game Detailed replay rank Binary-feedback rank
Pacman 1 11
AntWar 9 22
Miracle 15 23

Source: Appendix C, Table 7. Separate feedback-ablation conditions, not replacements for the main benchmark scores.

The Elo improvements are 526.0 points in Pacman, 238.1 in AntWar, and 67.1 in Miracle. Because each game has its own rating environment, those gains should be interpreted separately rather than averaged into a cross-game improvement measure.

A binary outcome tells the developer whether a policy succeeded. A detailed replay provides the states, actions, and consequences needed to investigate why it succeeded or failed. That additional diagnostic information offers a more direct basis for revising the code.

Opponent selection also matters. In the three-game ablation, model-directed selection outperforms random and top-four-only selection. However, a progressive ladder of increasingly strong opponents produces the best results in Pacman and AntWar, while model-directed selection performs best in Miracle.

Batch-size experiments reveal another tradeoff. Four opponents per request perform best in Pacman and Miracle, while two perform best in AntWar. Larger batches are not automatically more informative if they reduce opportunities to interpret feedback and revise a strategy between requests.

These experiments support a practical principle: the value of an interaction depends on the diagnostic information it supplies and how the agent uses that information. The exact testing schedule remains task-dependent.

Other players’ experience can help, but not consistently

An agent normally learns from matches involving its current program. The researchers also investigate whether it can improve by studying matches played between other programs.

They call this off-policy replay learning. Here, the term describes the source of replay experience, not a parameter-updating reinforcement-learning algorithm.

The comparison produces mixed outcomes. In Miracle, other-player replays yield rank 15 versus rank 26 using the agent’s own replays. Both approaches reach rank 1 in Pacman. In AntWar, the other-player condition reaches rank 19 versus rank 5 for own-policy experience.

External experience can therefore expose useful strategies beyond those encountered by the agent’s existing program. But different games benefit differently, and the compared policies emerge from independently developed runs.

For an autonomous software-improvement team, this suggests testing historical execution records as an additional source of evidence. It does not establish that externally collected traces should replace tests of the current program.

Designing a better autonomous improvement loop

The business opportunity is not demonstrated autonomous optimization of commercial software. It is a set of testable design choices for teams developing such systems.

Consider a team maintaining executable scheduling or allocation rules. A controlled improvement workflow could retain the following capabilities:

  1. Versioned candidates and validated champions. Keep experimental revisions separate from the best independently evaluated program. Record regressions rather than allowing every revision to become the new deployment candidate.
  2. Diagnostic traces. Preserve relevant inputs, decisions, intermediate states, and outcomes so the agent can investigate specific failures instead of relying entirely on pass/fail results.
  3. Selective test allocation. Include representative, difficult, and progressively challenging cases rather than repeatedly testing only the strongest or most convenient examples.
  4. Separate evaluation and development. Use diagnostic tests to improve candidates, but assess retained policies against a stable reference set and additional unseen cases before considering deployment.
  5. Compute accounting. Track tokens, calls, execution time, and achieved performance together.

The final point deserves attention. Although the configurations share nominal interaction budgets, total main-run token consumption ranges from 260.34 million for GPT6-sol to 1,454.07 million for Qwen 3.8—more than a fivefold difference.

The paper does not convert these figures into comparable monetary costs or wall-clock efficiency. Still, they demonstrate that equal test allowances are not equivalent computational budgets.

A team selecting an autonomous developer therefore needs to evaluate both the quality of its retained output and the resources consumed to produce it.

What the benchmark does not establish

AAArena measures development against frozen historical human-program pools. A rank-1 placement is a benchmark ranking relative to those programs and their fixed ratings. It does not prove optimal play, direct victory over the original human champion, or robustness against unseen opponents.

The comparison also does not isolate model capability from tooling. Opus5.5 uses Claude Code, while the other evaluated configurations use Codex. Differences in performance may reflect both the underlying model and its development environment.

The feedback ablations involve only three games and one model–harness configuration. The budget continuation involves four runs. Neither can establish universal behavior across autonomous coding systems.

Finally, the paper does not directly compare AHL with a matched system that updates model parameters through reinforcement learning. Its contribution is to demonstrate and measure adaptation through executable software revision, not to prove that this approach is universally preferable to model training.

These boundaries matter most when an organization moves from benchmark competition to software whose errors have operational consequences.

Improvement requires more than another iteration

AAArena shows that fixed-weight AI developers can produce competitive programs through sustained testing and code revision. It also documents regressions, persistent plateaus, and sharply different outcomes depending on the feedback available.

For organizations considering self-improving software agents, the central design question is therefore how to make each experiment informative and each accepted revision demonstrably better.

A larger iteration budget expands the opportunity to improve. Diagnostic feedback, deliberate test selection, retained policy versions, and independent evaluation determine whether that opportunity produces a stronger executable system.

The research establishes those relationships within a competitive-game benchmark. Translating them into reliable business operations remains a separate engineering and validation task.

Cognaptus: Automate the Present, Incubate the Future.


  1. Kaisen Yang and Qingle Liu and Kejin Wang and Yicheng Zhao and Jieming Li and Shenghan Zheng and Ruize Yang and Bojun Yang and Heng Gong and Xiang Gao and Lanyue Zhang and Kaiyu Zhong and Zhuo Liu and Shaoxuan Li and Chengxi Li and Yong Yan and Weixuan Zhang and Tianwei Luo and Situ Wang and Youjie Zheng and Sihan Zhao and Shengyuan Wang and Huan-ang Gao and Jiazheng Xu and Xiaohui Xie and Wentao Han and Hongning Wang (2026). Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition. arXiv:2610.12341. https://arxiv.org/abs/2610.12341 ↩︎