TL;DR for operators

Historical optimization runs may be more valuable than their final solutions. They contain records of which candidates were explored, how search populations changed, and which transitions improved the objective.

Jian Yao and colleagues investigate whether that experience can train a reusable optimizer. Their Algorithm2Model (A2M) system first imitates existing algorithms, then learns to improve its own search behavior. On continuous BBOB benchmarks, it beats one of its evolutionary teachers on 106 of 120 configurations.1

The operational opportunity is to turn accumulated search histories into an improvable optimization policy. Controlled ablations support both heterogeneous teacher experience and performance-driven refinement. A separate experiment demonstrates incremental integration of five optimizers, improving mean area over the convergence curve by 61.3%.

These findings justify investigating trajectory-based learning for recurring black-box optimization workloads. They do not establish a universally transferable optimizer or prove that offline training and inference costs are economically justified.

The search history contains more than the winning solution

Consider an optimization team repeatedly tuning parameters for a simulation-based process. Several algorithms have been deployed, each producing thousands of evaluated candidates before reaching its final solution.

Keeping only the best candidate discards information about the search process. Intermediate populations reveal which candidates were generated, how alternatives interacted, and when particular search behaviors improved performance.

The central question is whether a model can learn from these histories without permanently inheriting the limitations of the algorithms that generated them.

A2M provides a measurable result.

On 120 held-out continuous BBOB function–dimension configurations, it records 106 wins, 12 losses, and 2 ties against GTDE, an evolutionary optimizer used during its training. Both operate under a 10,000-function-evaluation budget, with 20 independent runs per configuration.

A2M’s average rank is 1.52, compared with GTDE’s 4.60; lower is better. It also wins 95 configurations against the learned optimizer POM and 112 against MetaDistill, with 24 and 8 losses respectively.

These are benchmark comparisons, not demonstrations of production performance. Nevertheless, they establish the paper’s central empirical finding: learning from an optimizer’s execution can produce a search policy that performs better than the source algorithm on unseen test problems.

How A2M moves beyond imitation

The paper calls its broader proposal Algorithm Modelization: representing iterative algorithms as learnable state-transition behavior.

A2M implements that proposal for population-based black-box optimization. Instead of following a fixed handcrafted update rule, it learns to generate candidate solutions from recently evaluated populations.

Its Transformer consumes the latest five generations, preserving individual histories while representing interactions among candidates and across time. Coordinate-wise features are aggregated using dimension-independent pooling, avoiding a fixed input size tied to the number of decision variables.

The generated search transition uses signed population attention. Unlike ordinary softmax attention, its interaction coefficients can be positive or negative, permitting both attractive and repulsive combinations of existing solutions. Learned coordinate-level crossover then combines proposals with their parents. Actual objective evaluations and greedy selection determine which candidates survive.

A2M therefore generates where to search next. It does not simply predict the final optimum or replace the objective function.

Training has two distinct stages.

Stage I: inherit search behavior. The continuous model learns from trajectories produced by GTDE, population-adapted GP-LogEI, and Projected-Lion, representing evolutionary, Bayesian, and gradient-based search. The training record contains 6,480 trajectories and 615,600 five-generation history–transition windows. Supervision covers both proposed search operators and candidate transitions.

Stage II: improve search performance. Teacher supervision is removed. The model trains on states generated through its own optimization process, using an objective that rewards one-generation improvement in mean population fitness while encouraging population diversity.

The second stage requires differentiable training objectives. It does not backpropagate through complete optimization histories or discrete acceptance decisions.

Crucially, this training occurs before deployment. During the standard benchmark tests, A2M’s parameters remain fixed, and the deployed optimizer uses objective-function evaluations rather than objective gradients.

The ablations identify what contributes to improvement

The authors test whether the architecture’s gains could arise from imitation alone.

They compare the full system against imitation-only and online-only variants. The full pipeline wins 110 of 120 continuous BBOB configurations against imitation-only training and all 120 against online-only training.

A separate teacher-diversity ablation holds the architecture and training procedures comparable while changing the teacher sources. The heterogeneous model wins 115, 120, and 88 configurations against evolutionary-only, Bayesian-only, and gradient-only variants, respectively.

These are component ablations, not additional independent demonstrations of broad generality. They support a specific mechanism: prior algorithm experience supplies search behavior that subsequent performance feedback can refine. Neither stage, in isolation, reproduces the full reported performance.

Strong transfer results have three different meanings

A2M achieves the best reported average rank in all six scenario–suite comparisons in Table I. But the underlying experiments test different forms of reuse.

Evaluation setting Reported result What actually transfers
Continuous BBOB and CEC Average ranks of 1.52 and 2.35 A trained continuous optimizer operates on held-out functions
Mixed-variable MV-BBOB and EOPCCV Average ranks of 1.77 and 1.57 The architecture is reused with separately trained model parameters
Expensive BBOB and CEC Average ranks of 1.06 and 1.45 A frozen continuous search model is combined with a test-time-adaptive predictor

The clearest fixed-model extrapolation is dimensional.

Continuous A2M trains at dimensions 20, 50, 100, and 200, then achieves the best reported mean objective value on 16 of 24 BBOB functions at the unseen dimension of 500. This demonstrates extrapolation within the tested benchmark family, not unrestricted scalability.

Mixed-variable optimization requires a different interpretation. A2M maintains continuous internal coordinates and projects selected values into permitted integer or categorical representations before objective evaluation. The architecture is unchanged, but the model is trained separately using mixed-variable trajectories.

Expensive optimization introduces another distinction.

With only 300 true objective evaluations, A2M generates candidates while an independent X-to-Y predictor ranks them. The protocol starts with 100 evaluations and allocates another 200 over 40 rounds, selecting three exploitation candidates and two diversity candidates per round.

The A2M search checkpoint remains frozen, but the predictor adapts using observations from the current problem. The excellent expensive-optimization ranks therefore belong to the combined search-and-evaluation system, not to a fully adaptation-free model.

For an optimization-platform developer, these distinctions determine whether an existing checkpoint can be deployed directly, whether a new training run is necessary, or whether an adaptive evaluation service must also be maintained.

Five optimizers become a progressively trained model

A strong pretrained optimizer might still become obsolete when better algorithms appear.

The paper’s continual-evolution experiment examines whether A2M can incorporate new optimization experience without restarting from every historical training dataset.

It sequentially introduces GA, jDE, GTDE, GP-LogEI, and Lion. Each successive training stage inherits the previous checkpoint and uses trajectories from the newly introduced algorithm, without replaying all earlier teacher trajectories.

Across the combined CEC2020 and CEC2026 evaluation, mean area over the convergence curve (AOCC) rises from 0.0794 to 0.1281, a 61.3% relative improvement. Pooled average rank improves from 4.571 to 1.469.

AOCC measures performance over the convergence process. The 61.3% result is not a reduction in final objective value, inference time, or operating expenditure.

This sequential experiment supports the possibility of accumulating optimization capability through checkpoint inheritance. It does not establish that knowledge will remain intact across indefinitely long training sequences, conflicting teachers, or major distribution shifts.

The operational test is broader than solution quality

Cognaptus inference: An organization repeatedly solving related, bounded, single-objective optimization problems could treat historical optimizer trajectories as training assets rather than retaining only final solutions.

For an optimization-platform operator, that changes what should be preserved: evaluated populations, transition histories, objective values, variable representations, and enough execution context to reconstruct useful training examples.

A practical evaluation would compare a trajectory-trained optimizer against existing search methods on previously unseen problems from the organization’s actual workload. Alongside fixed function-evaluation budgets, it should measure offline trajectory generation, model-training expenditure, inference latency, objective-evaluation cost, and performance under operational constraints.

Three boundaries matter.

First, the demonstrated training and evaluation focus on standardized functions and box-constrained, single-objective problems. They do not validate behavior on constrained, multi-objective, or unrestricted production optimization.

Second, equal function-evaluation budgets do not imply equal total computational costs. A2M introduces offline data generation and training, while its expensive-optimization configuration adds adaptive prediction infrastructure.

Third, the general Algorithm Modelization proposal extends conceptually beyond optimization, but the reported experiments do not establish that arbitrary algorithms can be converted into equally effective learned models.

These limitations change the deployment decision. The strongest immediate candidate is a recurring optimization service with sufficiently similar problem structures and enough repeated demand to justify the initial training investment.

Search experience is becoming an algorithmic asset

A2M’s contribution is not simply that another learned optimizer wins benchmark comparisons. Its more consequential result is that optimizer execution histories can supervise a search policy that subsequently improves beyond its teachers.

The two-stage ablations make that interpretation more credible. Dimensional extrapolation tests one form of reuse; mixed-variable and expensive-optimization experiments clarify where that reuse requires retraining or auxiliary adaptation. Sequential checkpoint inheritance provides an initial demonstration of cumulative capability.

For operators, the next question is measurable: does retaining and learning from search trajectories produce better future optimization outcomes after accounting for the full cost of doing so?

The paper supplies a credible benchmark-level reason to test that proposition. Production reliability and net economic value remain to be established.

Cognaptus: Automate the Present, Incubate the Future.


  1. Jian Yao and Zhiyao Zhang and Yichen Li and Xingyu Wu and Kay Chen Tan (2026). Algorithm Modelization: Transforming Optimization Algorithms into Evolvable Intelligence. arXiv:2610.23161. https://arxiv.org/abs/2610.23161 ↩︎