Scaling the Sandbox: When LLM Agents Need Better Worlds

Sandbox is a comforting word. It sounds safe, contained, childlike. Put an AI agent in a sandbox and let it practice. Nothing catches fire. Nobody accidentally cancels a real flight. No production database wakes up with 37 mysterious refund requests and a very confused compliance officer.

The problem is that most agent sandboxes are either too fake to teach anything, too manual to scale, or too close to production to be relaxing. The agent has to learn how to navigate persistent state, business rules, incomplete user information, tool failures, and multi-step dependencies. A static API-call dataset does not teach that. A role-playing LLM pretending to be the environment may hallucinate the rules. A hand-built benchmark is useful, but expensive to multiply.

That is the practical tension behind EnvScaler, a paper that proposes an automated pipeline for synthesizing executable, stateful, tool-interactive environments for LLM agent training.¹ The paper is not just another “we made synthetic data and the benchmark went up” story. The interesting part is the mechanism: EnvScaler tries to manufacture little programmable worlds, then generate tasks inside those worlds, then verify agent success by checking the final state of the world.

That last clause matters. A lot.

The real bottleneck is not the tool list, but the world behind it

Many tool-use systems treat tools as isolated functions. The model learns to call get_order_details, cancel_order, or update_address with the right parameters. That is useful, but it is not the same as being an agent inside an environment.

A real workflow is not a list of tools. It is a stateful system with rules. An order may be cancellable only before shipment. A return may require payment-method consistency. A calendar conflict may depend on attendees, time zones, and prior commitments. A document may exist, move, disappear, or be permission-locked. The agent does not merely select functions. It has to inspect state, infer missing information, call tools in a plausible order, react to feedback, and know when the state has become correct.

EnvScaler’s core move is to represent that world as executable code. The environment is not an LLM improvising a response. It is a Python program with state attributes, tool methods, and rules. The agent sees documentation and tool interfaces, calls tools, receives observations, and changes the environment state.

This is the paper’s first useful correction to a common misconception: synthetic agent training is not mainly about making an LLM role-play a customer, a database, or a backend system. That can create variety, but it also creates inconsistency. The paper’s bet is that the environment should be programmatic. The LLM may help write the environment, but once built, the environment behaves like software.

The distinction is small enough to miss and large enough to matter. A simulated environment that changes its mind is not a training ground. It is a polite hallucination wearing a badge.

EnvScaler has two machines: one builds worlds, the other builds missions

The paper’s mechanism-first structure is clean. EnvScaler has two major components:

SkelBuilder answers the question: what kind of world should exist, and how should it work?

ScenGenerator answers the next question: what should the agent do inside that world, and how do we know whether it succeeded?

The paper reports synthesizing 191 environments and roughly 7,000 task scenarios. Those environments are not toy one-function APIs. The reported average is 18.58 tools per environment, split between information-query tools and state-change tools, and 21.38 state categories per environment. The point is not that these are perfect enterprise twins. They are not. The point is that they are complex enough to force multi-step interaction instead of one-shot function calling.

SkelBuilder starts from tasks, then reverse-engineers the environment

SkelBuilder begins with existing task sets, then filters for tasks that imply a persistent, domain-specific, actionable environment. This is an important design choice. Instead of asking a model to invent random environments from the void—always a fine way to get “Smart Restaurant Booking Blockchain CRM Platform” by accident—it mines tasks that already imply operational settings.

From a task, the pipeline infers an environment theme. Then it asks an LLM to plan the state space, business rules, and tool operations. The environment is represented as a Python class: attributes hold state, methods implement tools, and documentation describes the environment.

The pipeline has three steps:

1. Task-guided environment discovery: find tasks that require stateful interaction, infer the environment behind them, deduplicate similar themes.
2. Executable environment construction: plan state, rules, and tools; convert them into runnable code.
3. Dual-agent environment assessment: stress-test the environment through tool calls, then judge whether the implementation behaves correctly.

That third step is where the paper becomes more serious. EnvScaler does not simply ask an LLM, “Does this environment look good?” It creates a testing loop. A frontend testing agent generates positive and negative tool calls. A backend checking agent inspects the method source code, the return value, and the before/after state transition, then labels the behavior as pass, warning, or fail.

The paper uses 100 rounds of such testing and keeps environments above a pass-rate threshold of 0.85. From 266 synthesized environments, 191 survive, implying a 28.2% rejection rate. The rejected environments mostly fail for mundane software reasons: type errors, attribute errors, and state inconsistency. Not glamorous. Very useful. Real engineering progress often looks suspiciously like cleaning up boring failure modes.

This quality filter is not a side note. It is part of the mechanism. Without it, “scalable environment generation” becomes “scalable generation of broken Python objects,” which is a less attractive headline.

ScenGenerator makes tasks state-dependent instead of sequence-dependent

Once an environment exists, the next problem is task generation.

A weak approach would be to generate a tool sequence first, then reverse-engineer a natural-language instruction from it. That makes the trajectory look supervised, but it also creates a hidden problem: there may be more than one valid way to solve the task. If the benchmark rewards matching one reference sequence, it can punish correct alternatives.

EnvScaler does something more useful. It generates an initial environment state, then creates a challenging task based on that state, and finally decomposes the task into verifiable conditions. Each condition becomes a validation function that checks the final environment state.

This is the hinge of the paper.

The agent is not rewarded because it called the same functions as the teacher. It is rewarded because the final world state satisfies the task’s conditions. If the task is to resend a corrected message, archive the relevant conversation, delete an old failed message, and mark another message as read, then the validation functions can check those final facts directly.

That has three consequences.

First, it allows multiple solution paths. Second, it supports partial credit, because each checkpoint can pass or fail independently. Third, it gives RL a concrete reward signal grounded in state, not in surface imitation.

For enterprise agent training, this is much closer to how operational success should be judged. Nobody should care whether the agent used the exact same internal route as yesterday’s support agent. They should care whether the refund was issued correctly, the customer was notified, the exception was logged, and the policy was not violated. Revolutionary concept: measure the job, not the choreography.

The paper tests whether these worlds teach transferable tool behavior

The experimental design has a clear main question: do agents trained in EnvScaler’s synthetic environments perform better on unseen multi-turn, multi-tool benchmarks?

The authors train Qwen3-series models using supervised fine-tuning and reinforcement learning. For SFT, they use 140 environments and generate about 9,000 final trajectories using a stronger teacher model. The remaining 51 environments are used for RL. Evaluation is conducted on three tool-agent benchmarks: BFCL-v3 Multi-Turn, Tau-Bench, and ACEBench-Agent.

The headline result is that EnvScaler improves performance across model sizes and benchmarks, but not uniformly. That unevenness is where the article becomes more interesting.

This table is the right way to read the experiments. The paper does not contain one monolithic proof. It contains a stack of evidence: main gains, then checks on similarity, scaling, interaction style, RL dependence, environment quality, and cost.

The main numbers say SFT is the reliable engine; RL is the selective amplifier

The most stable result is SFT.

Averaged across three Qwen3 models, supervised fine-tuning with EnvScaler improves BFCL-MT by 8.67 points, Tau-Bench by 4.29 points, and ACEBench-Agent by 11.57 points. That pattern makes sense. SFT teaches reusable patterns: inspect state before acting, ask for missing parameters, sequence tool calls, and stop when enough state has changed.

For Qwen3-4B, BFCL-MT overall rises from 25.38 to 34.88 with SFT, then to 38.00 after SFT plus RL. Tau-Bench rises from 33.44 to 38.20, then 41.06. ACEBench-Agent rises from 55.28 to 66.67, then 70.55.

For Qwen3-8B, BFCL-MT rises from 28.88 to 37.00 with SFT, then 41.88 after RL. Tau-Bench moves from 38.19 to 41.35, then 44.81. ACEBench-Agent rises from 60.00 to 71.67, then 72.50.

That final increment is revealing. RL helps, but it does not behave like magic dust. The paper explicitly notes that RL is more model-dependent. Larger models exploit the synthetic environments better. Smaller models are more vulnerable to sparse or noisy rewards and weaker exploration.

This matters for business interpretation. If a company hears “synthetic environments plus RL,” the tempting conclusion is that it can throw a small model into a simulator and wait for competence to emerge. The paper does not support that fantasy. A more careful interpretation is: SFT provides the broad behavioral foundation; RL can refine strategies when the model has enough base capability to explore productively.

That is less cinematic. It is also more useful.

The similarity test says the agent is not merely memorizing domains

One of the paper’s better tests examines whether performance depends on similarity between training environments and test environments. The authors compute similarity using environment topic and toolset descriptions, then train on the most similar 50%, least similar 50%, and a random 50%.

For Qwen3-4B on BFCL-MT, the baseline overall is 25.38. Full SFT reaches 34.88. Training on the top 50% similar environments gives 32.00. Training on the bottom 50% gives 32.50. Random 50% gives 31.25.

This is not the main evidence; it is a robustness-style check against the lazy explanation that the synthetic environments simply resemble the benchmark environments. The result suggests that some transferable behavior is being learned: how to operate in stateful, tool-mediated worlds, not just how to handle one familiar domain.

Do not overread it. The similarity metric is based on text embeddings of descriptions and toolsets, not a deep causal map of all business rules. But the direction is useful. It suggests that environment diversity can teach procedural patterns that travel across domains.

That is the beginning of a business case for synthetic environment libraries. A retailer, insurer, clinic, or logistics firm may not need one perfect simulator for every workflow before seeing benefit. They may need a sufficiently varied set of stateful workflow worlds that teach agents how to behave when tools, rules, and missing information interact.

Scaling helps, but the first environments matter most

The scaling experiment studies what happens as the number of SFT training environments increases. The paper reports a steady upward trend for Qwen3-4B on BFCL-MT and ACEBench-Agent, with the sharpest improvement when moving from 0 to 20 environments. Gains continue beyond that, but more slowly.

This is another result with practical weight. It implies that environment scaling has diminishing returns, but not immediate saturation. For a business team, the first question is not “can we synthesize 10,000 workflow worlds?” It is more likely “which first 20 to 50 environments cover the operational patterns our agents keep failing?”

That reframes environment design from an academic scaling race into a product-management exercise. The useful unit is not “one more synthetic task.” It is one more distinct workflow pattern: missing customer data, conflicting policy constraints, multi-record updates, permission boundaries, rollback after failure, escalation after rule violation, cross-system reconciliation.

The paper’s environment count is still modest by production standards. Around 200 environments is not the universe. But it is enough to show a slope.

Interaction style is not cosmetic training data

EnvScaler generates two interaction settings.

In the Non-Conversation setting, the agent receives the full task at the start and acts through tools until completion. In the Conversation setting, an LLM-simulated user reveals information progressively, so the agent must talk, ask, and act.

The interaction-pattern ablation is useful because it prevents a common mistake: treating multi-turn dialogue as a stylistic wrapper around tool use. It is not. It changes the task.

In the BFCL-MT experiment with Qwen3-8B, Non-Conversation SFT performs better on Base and Long-Context subsets, where the necessary task information is already available. Conversation SFT helps more on Missing-Parameter tasks, where the agent must obtain absent information from the user. The combined setting reaches the best overall score: 37.00, compared with 35.75 for Non-Conversation only and 35.50 for Conversation only.

The paper also notes that Missing-Function performance does not improve in the same way because the training data does not contain missing-tool-type samples. That is an excellent little boundary condition. It says the model learns the interaction patterns it sees. It does not spontaneously develop a grand philosophy of unavailable APIs. Sad, but familiar.

For enterprise design, this means training data should not only include “successful workflow execution.” It should deliberately include information-gathering episodes, missing-tool episodes, unavailable-record episodes, user-correction episodes, and cases where the right answer is to stop or escalate.

The quality analysis is more than housekeeping

EnvScaler’s quality analysis evaluates synthesized environments across three dimensions: tool-program alignment, functional correctness, and code robustness. The reported average scores are above 8.5 across all three: 8.58 for tool-program alignment, 8.97 for functional correctness, and 8.60 for program robustness.

This is not the proof that the environments are business-realistic. It is evidence that the generated software artifacts are often internally usable after filtering.

The discarded-environment analysis is equally important. Type errors, attribute errors, and state inconsistency are the leading failure categories. That is exactly what one would expect from LLM-generated code assembling stateful tools. It also tells us where the automation stack needs reinforcement: stronger schema checking, static typing, state-invariant tests, boundary-condition tests, and maybe less faith in “the model wrote a docstring, therefore the universe is aligned.”

For a business team, quality assessment should not be treated as optional polish. Synthetic sandboxes can only train reliable agents if the sandbox rules themselves are reliable. A wrong simulator teaches wrong behavior efficiently. Efficiency is not always your friend.

The cost numbers are small enough to be interesting, not final enough to be comforting

The paper reports an average cost of about $1.024 per environment\ast\ast and \ast\ast$0.0635 per scenario under a GPT-4.1 / GPT-4.1-mini combination, measured on a small batch. The major cost driver is the dual-agent assessment loop, which consumes far more tokens than discovery or construction.

These numbers are useful because they suggest the approach is not obviously absurd. Generating hundreds or thousands of sandbox scenarios may be economically feasible.

But the cost table should be read as an implementation detail, not a total-cost-of-ownership estimate. Real enterprise use would add domain expert review, data governance, simulator maintenance, integration with workflow logs, security evaluation, privacy controls, and repeated regeneration as policies change. The cheap part may be token generation. The expensive part is knowing whether the synthetic business rule is actually the rule.

Still, a $1-ish environment-generation cost is strategically provocative. If the bottleneck moves from “can we afford to create sandbox tasks?” to “can we specify and validate the right worlds?”, then the competitive advantage shifts toward organizations that understand their own operations well enough to simulate them.

That is a less glamorous bottleneck than model architecture. It is also where many companies quietly lose.

What businesses can infer, and what they cannot

The paper directly shows that programmatically synthesized environments can improve tool-agent benchmark performance for Qwen3 models, especially through SFT and especially on benchmarks emphasizing multi-turn, multi-tool behavior. It also shows that state-based validation can provide rewards for RL and that environment diversity appears to support transfer beyond near-similar domains.

Cognaptus would infer a practical pathway from this:

This pathway is not “buy EnvScaler, automate the company.” Please, let us remain adults. The right interpretation is that enterprise agent development may need an internal simulator layer: a library of executable business worlds where agents can practice before they touch real systems.

That simulator layer could become as important as prompt engineering, retrieval design, or model selection. It is where policy, process, tools, and data state become trainable structure.

Boundaries: EnvScaler builds sandboxes, not full cities

The authors’ limitations are practical and important.

First, although EnvScaler does not use an LLM as the runtime simulator, it still uses LLMs to synthesize the environment logic. That can introduce bias or incorrect assumptions in business rules, state definitions, and task design. Programmatic does not automatically mean correct. It means inspectable and executable.

Second, the environments are mostly domain-specific and stateful. Open-ended web search and broad information-access tasks are not the main target. This matters because many real agents combine workflow tools with open-world research. EnvScaler handles the workflow side more naturally than the open-world side.

Third, the environments focus on tool-state interaction. They do not explicitly model latency, network instability, UI friction, rate limits, authentication weirdness, partial outages, or the deeply human joy of enterprise software returning an error message that means nothing and everything.

Fourth, the setup is text-only. Multimodal tools involving images, audio, documents with layout, screenshots, and visual interfaces remain outside the main scope.

Finally, the current experimental scale is about \ast\ast191 environments\ast\ast and \ast\ast7,000 scenarios\ast\ast. That is impressive for a research pipeline, but still small relative to the operational diversity of a large enterprise.

These boundaries do not weaken the paper’s core idea. They prevent the wrong business conclusion. EnvScaler is not a finished recipe for production-grade digital twins. It is a strong argument that executable synthetic environments are a missing training substrate for agents.

The deeper lesson: agents need curriculum, not just access

The current article this replaces made a clean point: bigger brains need bigger worlds. That still holds, but the paper is more precise than that slogan.

The lesson is not simply “scale environments.” It is: scale \ast\astexecutable, stateful, assessable\ast\ast environments; generate tasks tied to initial state; evaluate success through terminal-state checks; use SFT for stable behavioral transfer; use RL carefully, because exploration capacity matters.

That is a more demanding view of agent development. It asks organizations to stop treating tools as buttons and start treating workflows as worlds. A good agent does not merely know which API exists. It understands, through training and feedback, how state changes under rules.

For businesses, this may become the difference between demos and deployable agents. A demo can survive on a clever prompt and a few polished tool calls. A deployed workflow agent needs to practice in a world where mistakes have consequences, but not real ones. The sandbox must be cheap enough to scale, strict enough to teach, and transparent enough to debug.

EnvScaler is valuable because it points toward that middle ground. Not real production. Not hallucinated simulation. Programmable practice worlds.

A sandbox, finally, with rules.

\ast\astCognaptus: Automate the Present, Incubate the Future.\ast\ast