Quants With a Plan: Agentic Workflows That Outtrade AutoML

TL;DR for operators

A quant team does not need a chatbot that “has ideas” about markets. It needs a workflow that can select a sensible model, change one thing at a time, run the experiment, keep the better version, reject the worse one, and leave a paper trail that a human can inspect without requiring divination.

That is the practical value of TS-Agent, a framework for financial time-series forecasting and synthetic generation proposed by Ang et al.¹ The paper’s important claim is narrower, and therefore more useful, than the usual “LLMs beat AutoML” headline. TS-Agent performs well because the LLM is not allowed to wander around the modelling problem like an intern with terminal access and too much confidence. It is constrained by a case bank, a financial time-series code base, a refinement knowledge bank, execution feedback, and logged chain-of-code-edits.

For operators, the lesson is simple: if you want agents in financial modelling, do not start with autonomy. Start with reusable rails. Put approved models, metrics, tuning practices, and audit logs around the agent. Then let it search inside that box.

The paper reports strong benchmark results across crypto, foreign exchange, and stock datasets. In forecasting, TS-Agent’s best LLM variants beat AutoGluon on RMSE by roughly 8% on Crypto and more than 20% on Exchange, while also beating generic agent baselines such as DS-Agent and ResearchAgent on most reported metrics. In generation, TS-Agent is competitive with or better than Optuna and is much more consistently successful than the generic agents across LLM backbones.

The business inference is not “replace your quant team.” Please, spare everyone. The inference is that agentic modelling becomes operationally interesting when it looks less like free-form AI creativity and more like governed experimental automation.

The mistake is treating financial modelling as generic search

AutoML is useful because it turns many modelling tasks into systematic search: choose preprocessing, select model families, tune hyperparameters, evaluate, repeat. That is already better than artisanal spreadsheet sorcery. But financial time series are a poor place to pretend that all modelling objectives are interchangeable.

A low RMSE forecast can still be useless if it distorts tail risk. A synthetic generator can match marginal distributions while mangling autocorrelation. A model that performs well on one market regime may quietly fail when volatility changes. And in regulated financial settings, the final metric is only half the story. Someone still has to explain how the model was chosen, what changed, why it changed, and whether the change was reviewed.

This is where the paper’s framing is stronger than a standard “new benchmark winner” claim. TS-Agent treats financial time-series modelling as a decision process with memory, constrained actions, domain resources, and feedback. The LLM is the planner, not the entire modelling system.

That distinction matters. A generic agent asked to “build the best model” has to generate code, infer evaluation choices, debug failures, select architectures, and justify decisions, often while mixing all of those concerns into one enormous soup. TS-Agent decomposes the work. It narrows model choice through past cases, edits a scaffolded train.py, applies refinement heuristics, tunes hyperparameters, executes the result, logs outcomes, and uses those outcomes to guide the next step.

The modelling problem therefore shifts from “Can the LLM invent a good quant pipeline?” to “Can the LLM orchestrate approved modelling components better than static search?” That is a much less glamorous question. Conveniently, it is also the one financial institutions can actually use.

What TS-Agent changes is not the LLM, but the workflow

The paper describes TS-Agent as operating through model pre-selection and code refinement. The abstract also describes the process as model selection, code refinement, and fine-tuning; in the framework section, refinement and fine-tuning sit inside the second stage. The useful operational picture is this:

Task description
      ↓
Case-based model pre-selection
      ↓
Round-robin refinement across candidates
      ↓
Hyperparameter fine-tuning
      ↓
Execution, metrics, logs
      ↓
Memory update and next edit

The crucial move is that the agent is not asked to solve the whole modelling problem in one heroic pass. It receives a task, a scaffolded Python training script, and three external resources:

This is the part of the paper worth slowing down for. The reported performance gains are interesting, but the mechanism is the exportable idea. TS-Agent is not merely a smarter search algorithm. It is a modelling workbench where the agent’s freedom is channelled through approved components.

In finance, that matters because uncontrolled flexibility is not a feature. A system that can generate any code can also generate any mistake. The paper’s design tries to shrink the dangerous surface area. Model selection is constrained by the case bank and model bank. Fine-tuning is constrained by configuration choices. Refinement is the area most likely to introduce bugs, so the framework isolates it and logs changes.

That is the adult version of agentic AI: not “the agent can do anything,” but “the agent can do specific things, in specific places, with evidence attached.” Less cinematic. More useful.

The three banks are the control surface

TS-Agent’s knowledge banks should not be read as decorative RAG furniture. They are the control surface of the system.

The Case Bank gives the planner a memory of task-model fit. A stock forecasting problem with a 60-day input window and a three-day prediction horizon need not be treated as a blank page. Similar tasks can suggest candidate architectures before the agent wastes iterations on unsuitable options.

The Code Base turns model development from freehand code generation into component reuse. For forecasting, the experiments include representative models such as Autoformer, PatchTST, TimesNet, DLinear, and TimeMixer. For generation, the model bank includes GAN-based models, a VAE-based model, and a diffusion-based model. The point is not that these are the only possible models. The point is that the agent edits around known implementations rather than hallucinating a fresh research codebase because it woke up feeling Bayesian.

The Refinement Knowledge Bank is where financial modelling practice enters the loop. It includes guidance on scaling, normalisation, data augmentation, early stopping, adaptive learning rates, dropout, weight decay, cross-validation, overfitting diagnostics, and leakage handling. These are not glamorous. They are exactly the kinds of boring practices that prevent expensive nonsense.

For business teams, this suggests a useful deployment principle: the first implementation work is not building an omniscient agent. It is curating the banks. A bank, fund, or fintech team would need to encode its approved model families, data split policies, validation rules, leakage checks, risk metrics, and review requirements before expecting the agent to help. Otherwise, the “agentic workflow” is just prompt engineering wearing a lab coat.

Chain-of-code-edits turns experimentation into evidence

The paper’s nicest phrase is “chain-of-code-edits,” an analogy to chain-of-thought, except more operationally grounded. Instead of merely recording reasoning text, TS-Agent records the sequence of code changes, the rationale for each change, the execution result, and the training metrics.

This matters because modelling improvement is usually a path, not a point. A final model artefact tells you what won. It often does not tell you what was tried, what failed, or whether a suspiciously good result came from a legitimate improvement or a leak wearing a fake moustache.

TS-Agent’s refinement loop works more like a controlled search over edits. If a candidate script improves the loss, it replaces the incumbent. If it does not, the system reverts to the previous script. In the warm-up phase, multiple shortlisted models receive a smaller number of refinement iterations. The best model-refinement-hyperparameter combination is selected. In the optimisation phase, the chosen candidate receives more iterations.

That is not a full substitute for model risk governance. It is, however, much closer to the kind of process governance teams can inspect:

The important business shift is from answer production to experiment management. The agent is valuable not because it produces a single impressive output, but because it can run a sequence of bounded experiments while preserving enough context for humans to understand the path.

The experiments test full-system advantage, not component causality

The paper evaluates TS-Agent on two families of tasks: forecasting and synthetic generation. The datasets cover Crypto, Exchange, and Stock settings. Crypto uses hourly closing prices for 20 cryptocurrency trading pairs against USDT in 2024. Exchange uses daily foreign exchange rates from 1990 to 2010 for eight major currency pairs quoted against the U.S. dollar. Stock uses daily closing prices for 10 major U.S. stocks from January 2020 to December 2024.

The baselines are sensible for the claim being tested. For forecasting, the paper compares against AutoGluon. For generation, it compares against Optuna. It also compares against two agentic systems, DS-Agent and ResearchAgent. Each agent is instantiated with four LLM backbones: GPT-3.5, GPT-4o, Claude Sonnet 4, and Nova Pro. Metrics are averaged over five runs.

That design mainly tests whether the full TS-Agent workflow performs better than generic AutoML and generic agentic baselines under the authors’ benchmark setup. It does not isolate the contribution of every component. There is no clean ablation saying, for example, “remove the Case Bank and performance drops by X,” or “remove chain-of-code-edits and success rate falls by Y.” The ranking figures are comparison summaries, not causal component tests. The case study is an implementation and auditability illustration, not a second benchmark thesis.

That boundary does not weaken the paper. It clarifies how to read it.

This distinction is important for operators. The paper is not saying, “Install TS-Agent and mint alpha.” It is saying that structured agentic workflows can produce more accurate and more reliable modelling results than less constrained alternatives in a benchmarked financial time-series setting.

That is already enough to be interesting. No need to put a cape on it.

What the forecasting results actually show

For forecasting, the paper evaluates RMSE, MAE, MAPE, sMAPE, and success rate. On the Crypto dataset, the best TS-Agent RMSE is 0.206 with GPT-4o, versus 0.223 for AutoGluon. That is roughly a 7.6% reduction. On Exchange, the best TS-Agent RMSE is 0.0068, again with GPT-4o, versus 0.0089 for AutoGluon, a reduction of roughly 23.6%. On Stock, the best TS-Agent RMSE is 7.982 with Claude, versus 8.430 for AutoGluon, a smaller but still favourable reduction.

The success-rate pattern is also notable. TS-Agent reaches 100% success for all LLM backbones on Crypto, all but Nova Pro on Exchange, and all but GPT-3.5 on Stock. DS-Agent and ResearchAgent are more variable. That matters because production teams care about repeatability, not merely the best lucky run. “It worked once in the notebook” is not an operating model. It is a cry for help.

The Crypto risk/trading metrics add a useful finance-specific angle. The paper reports differences between prediction-implied and ground-truth Sharpe Ratio, VaR, and Expected Shortfall. Lower is better. With GPT-4o, TS-Agent records a Sharpe Ratio Difference of 0.378 and VaR Difference of 0.0069, compared with 0.460 and 0.0089 for DS-Agent, and 0.471 and 0.0086 for ResearchAgent. AutoGluon’s Sharpe Ratio Difference is 0.402 and VaR Difference is 0.0076.

The interpretation is not that TS-Agent is now a trading strategy. The interpretation is that its forecasts better preserve some risk-sensitive structures in the tested Crypto setup. That is more useful than a pure point-error win because finance teams rarely deploy forecasts as decorative numbers. Forecasts usually feed position sizing, hedging, risk limits, liquidity planning, or stress assumptions. If the forecast gets the point estimate slightly right but the risk profile wrong, congratulations, the dashboard is pretty and the portfolio is on fire.

The generation results are about synthetic fidelity and reliability

The generation task is different. Here the goal is to create synthetic financial time series that match real-data structure. On Exchange and Stock, the paper evaluates marginal distribution, correlation, autocorrelation, and covariance differences. On Crypto, it focuses on VaR and ES differences because crypto returns are heavy-tailed.

The results are strongest on consistency. TS-Agent reaches 100% success across all four LLM backbones in the Exchange, Stock, and Crypto generation tables. DS-Agent and ResearchAgent vary widely, including zero-success cells where no successful runs are recorded. That alone is meaningful for synthetic data workflows. A generator that sometimes works, sometimes fails, and occasionally returns nothing is not a risk simulation system. It is a suspense genre.

On Exchange generation, TS-Agent’s best Correlation score is 1.874 with Nova Pro, compared with 2.114 for Optuna. On Stock generation, TS-Agent’s best Correlation score is 1.194 with Claude, compared with 1.530 for Optuna. On Crypto generation, TS-Agent’s VaR Difference is competitive with Optuna: the best TS-Agent value is 0.00205 with GPT-3.5, compared with 0.00220 for Optuna. For ES Difference, Optuna’s 0.000832 is better than some TS-Agent backbones, while TS-Agent with Nova Pro reaches 0.000566 and Claude reaches 0.000629.

So the sober reading is this: TS-Agent is not uniformly dominant on every metric cell. It is consistently strong, often best, and much more reliable than the generic agents in the reported setup. It also competes well with Optuna while doing more than hyperparameter tuning, because it selects model families and refinement strategies too.

For financial institutions, the synthetic generation angle may be as important as forecasting. Synthetic time series are useful for stress testing, data augmentation, privacy-sensitive experimentation, and scenario analysis. But a synthetic generator must preserve temporal dependence, cross-asset relationships, and tail behaviour. Otherwise, the synthetic data becomes a beautifully formatted lie. TS-Agent’s metric bank pushes the workflow toward those finance-relevant properties rather than treating generation quality as one generic loss number.

The case study shows how an agent learns without pretending to be a quant genius

The paper’s case study uses a U.S. stock forecasting task: predict the next three trading-day closing prices for 10 major companies using the past 60 business days of data, evaluated by average MAPE. TS-Agent starts with a scaffolded train.py containing placeholders for the dataset loader, model, and training loop.

In Stage 1, the agent shortlists Autoformer and PatchTST through case-based reasoning. In Stage 2, it runs the warm-up and optimisation process. The logs show plan summaries, execution results, and metric movements. Autoformer reaches MAPE 3.41 in warm-up; PatchTST reaches 4.17. Autoformer is then selected for further optimisation, where iterative refinements reduce MAPE to 1.86.

The result is useful, but the path is the point. The case study shows the agent trying changes such as learning-rate scheduling and weight decay, observing whether they help, rejecting non-improving directions, and retaining better variants. It resembles a junior quant following a disciplined experiment protocol. That is meant as praise. The alternative junior quant is doing unsupervised architecture improvisation at 1:00 a.m. and naming the file final_final_v7_really.py.

For managers, this changes the review conversation. Instead of asking, “Why did the model change?” and receiving a final artefact with vibes attached, reviewers can inspect a sequence of edits, rationales, and results. That does not eliminate the need for human review. It gives review something to review.

The business value is governed acceleration, not magic alpha

TS-Agent is most relevant for organisations that already have modelling assets: existing code, benchmark tasks, validation standards, model inventories, and internal heuristics. The framework assumes these resources can be made available as read-only external banks. That is an important assumption. A firm with no modelling discipline cannot sprinkle an LLM over chaos and get governance. It will get faster chaos. Very efficient, very modern, still chaos.

The practical adoption path is therefore not to deploy TS-Agent straight into production. It is to build a constrained pilot around one recurring workflow.

A realistic implementation would look like this:

This is where the ROI may sit. Not in replacing research judgement, but in reducing the cycle time between modelling hypothesis, implemented experiment, metric feedback, and reviewable change. A quant team could use such a system to run more disciplined candidate experiments. A model risk team could use the logs to understand how a candidate evolved. A product team could use the framework to maintain repeatable forecasting pipelines across assets.

The agent is not the product. The governed workflow is.

What remains uncertain before anyone uses this seriously

The paper’s boundaries are straightforward and worth stating once, cleanly.

First, the experiments are benchmark experiments. They are not live trading trials, and they do not prove net profitability after transaction costs, slippage, capacity constraints, market impact, or risk limits. Forecast quality and trading utility are related. They are not married.

Second, each metric is averaged over five runs. That is useful for consistency checks, especially success rate, but it is not a large-scale operational reliability study. Production systems fail in more creative ways than benchmarks. They have stale data, vendor changes, schema drift, holidays, corporate actions, missing timestamps, permission problems, and that one cron job nobody owns.

Third, the paper tests the full TS-Agent workflow against baselines. It does not provide a detailed ablation of each component. The authors argue that the code base and modular template reduce variance, and the evidence is consistent with that argument. But the paper does not isolate the marginal effect of each bank, each feedback mechanism, or each logging element.

Fourth, the framework depends on curated resources. This is not a weakness; it is the design. But it means performance and governance quality will depend heavily on the quality of the Case Bank, Code Base, Refinement Knowledge Bank, and evaluation metrics. A stale model bank can trap the agent in yesterday’s assumptions. A weak metric bank can optimise the wrong behaviour. A sloppy case bank can retrieve bad precedents with impressive confidence. Finance has enough of those already.

Fifth, auditability is not the same as approval. Logs help explain what happened. They do not certify that the modelling process satisfies a particular regulator, internal model-risk standard, or fiduciary requirement. Human governance remains in the loop, ideally before the model meets client money.

The operator’s checklist

The paper points toward a practical checklist for any team considering agentic modelling in finance:

This is the real “agentic” lesson. Autonomy without structure is not intelligence. It is liability with a nice interface.

Conclusion: the best quant agent is a planner on rails

TS-Agent is interesting because it refuses the lazy version of agentic AI. It does not ask an LLM to become a quant researcher by force of prompt length. It gives the LLM a structured role: retrieve relevant cases, choose from approved models, refine scaffolded code, tune configurations, execute experiments, log decisions, and update the next step from feedback.

That is why the paper’s benchmark wins matter. They are not merely scoreboard entries against AutoGluon, Optuna, DS-Agent, and ResearchAgent. They suggest that financial modelling agents become more reliable when they are less free.

For business teams, the headline is not “LLMs can outtrade AutoML.” That phrase is catchy, but it smuggles in more certainty than the evidence supports. The better conclusion is this: in financial time-series work, agentic systems become useful when they behave like disciplined modelling workflows, not like autonomous geniuses.

Give the agent rails, memory, metrics, and logs. Then let it move fast.

Just not unsupervised in production. We are building financial infrastructure, not a reality show.

Cognaptus: Automate the Present, Incubate the Future.