The Catalog Grew. The Agent Needed a Call Stack.

TL;DR for operators

As an agent’s tool catalog grows, it must solve two linked problems: choosing the right capability without carrying every tool schema into each decision, and remembering where to return after several nested actions. The paper’s hierarchy addresses both by showing the model only the options relevant to its current branch and storing nested workflow state explicitly.

The context advantage is substantial in the tested scale-up benchmark. With 128 tools and 800-character schemas, flat routing used 28,178 input tokens in one model call, while hierarchical routing used 1,733 across four calls. The hierarchy therefore reduced schema exposure enough to preserve context feasibility, but it replaced one broad decision with several sequential ones.

That trade-off matters because smaller prompts do not automatically produce better routing. When top-level domains overlap, an early branch choice can hide the correct tool; strict output-validation failures can also offset the architectural benefit. Teams should therefore treat hierarchy as an investment in orchestration, not merely catalog organization: it requires governed taxonomies, deterministic validation, latency measurement, and execution traces that preserve the path back to each parent task.

A growing tool catalog creates two operational failures

An agent that can call ten distinctive tools is usually manageable. An agent that can call hundreds of payment, fraud, support, data, and workflow tools faces a different problem: every decision may require carrying a large registry of descriptions and argument schemas. The prompt expands even when most tools are irrelevant to the current request.

Tool selection is only half of the difficulty. Once the agent delegates to a subtask, which invokes another capability, which then returns a result, the system must preserve where it came from, what the parent was waiting for, and which intermediate outputs belong to that branch. A global conversation transcript contains some of this information, but it does not provide an explicit execution model.

A Formal Hierarchical Architecture for Agentic Orchestration with Stack-Based Execution and Lazy Discovery addresses both failures through one runtime design.¹ Capabilities are arranged as a rooted tree. Internal nodes make routing decisions; leaves execute deterministic logic. The runtime stores the active path in a last-in, first-out stack and exposes only the current node’s immediate children to the model.

That combination is more consequential than organizing tools into folders. It changes both the visible action space and the representation of execution state.

Local discovery changes prompt scaling

A flat router places the global capability registry into each routing prompt. In simplified form, its context footprint grows with the total number of capabilities $N$ and their average schema length $S_{avg}$.

The hierarchical design instead exposes a bounded local set of children at each depth. For an approximately balanced tree with branching factor $k$, the paper models the path-level schema cost as:

The practical mechanism is straightforward: the model does not need to see every tool at every decision. It receives a local menu, chooses a branch, and then sees the next local menu. The total catalog can expand without proportionally expanding every prompt, provided the branching factor remains bounded and the tree is reasonably balanced.

The scale-up benchmark is the paper’s main empirical support for this claim. It holds the model, temperature, JSON protocol, validation schema, and scoring method constant while changing the routing architecture. With 800-character schemas and 128 tools, flat routing used 28,178 input tokens and 17,791 visible schema tokens in one call. Hierarchical routing used 1,733 input tokens over four calls, with 277 schema tokens visible per call.

At 256 and 512 tools, the tested flat prompts exceeded the context limit and recorded zero success. The hierarchy retained 1.0 routing success at both sizes, using 2,147 and 2,875 input tokens respectively. This is strong support for context feasibility within the tested configuration. It does not establish that every hierarchical implementation will be cheaper end to end, because the hierarchy performs multiple sequential calls and the study uses one model and inference stack.

The stack separates execution control from conversation memory

The second contribution is an explicit state model for nested work. The runtime represents global orchestration state as conversation state, a stack of active skill frames, and a pending-call pointer. Each frame stores the skill identity, its parent, its own output, and a localized accumulator for child outputs.

When an internal node delegates, its frame is pushed onto the stack and the child becomes the pending call. When a leaf finishes, its result is returned to the requesting parent frame. The runtime then restores that parent through explicit push, peek, and pop transitions rather than relying on the language model to infer where execution should resume.

The deterministic part is the return mechanism, not the entire agent. Routing decisions and completion signals remain probabilistic. This distinction prevents an overly broad claim: the architecture makes state restoration reconstructable and auditable, but it does not remove model error from branch selection or task completion.

For operations teams, explicit frames create a more useful execution trace than a single accumulating transcript. A failed workflow can be inspected by branch, parent-child relationship, pending call, and local output. Localized accumulators also restrict intermediate results to the requesting branch, which may reduce cross-domain interference and accidental propagation of malformed tool output. The paper presents this as a structural information-flow benefit, not as proof of prompt-injection immunity.

Smaller prompts do not guarantee better routing

The diagnostic benchmarks show where the architecture’s advantage ends.

Under noisy two-step workflow pressure, both flat and hierarchical systems converged near 0.73 task success from 32 to 128 tools. At 128 tools, however, the flat prompt reached 140,335 characters while the hierarchical prompt was 5,302. The hierarchy recorded no observed wrong-action or wrong-choice errors in that table, with residual errors concentrated more narrowly in arguments. This robustness test shows that hierarchy can substantially reduce prompt burden and change the error profile without improving the final success rate.

The semantic-overlap test is more severe. While the full catalog still fit, flat routing achieved 1.0 success across 16 to 128 tools. Hierarchical success ranged from 0.67 to 0.83. An early wrong choice removed the correct branch before the model could inspect leaf-level evidence. The smaller decision space became a liability because the taxonomy compressed distinctions too early.

A clean-semantic variant then made top-level domains mutually exclusive. Wrong-branch errors fell to zero, which supports the paper’s diagnosis that domain overlap drives part of the failure. Yet hierarchical success still declined from 1.0 at 16 tools to 0.77 at 128, with output-validation failures increasing. The result separates two implementation requirements: the tree must express meaningful boundaries, and the model must return option names and completion signals in exactly the accepted format.

The production-adjacent UPI suite should be read as an exploratory extension rather than a controlled architectural comparison. The observed hierarchical implementation reached 0.93 route-level success versus 0.87 for the flat terminal baseline, but the systems differed in coverage and runtime behavior. Boundary queries remained weak for both, with hierarchy at 0.40 and the flat baseline at 0.60. These results identify deployment cases worth testing; they do not establish end-to-end business-task superiority.

What enterprises should build around the hierarchy

The paper directly supports three design choices: bounded schema exposure, explicit nested state, and declarative capability registration. Cognaptus infers that these can improve enterprise control when tool catalogs are large, schemas are rich, and workflows cross several operational domains.

Implementation should begin with taxonomy governance, not tree construction. Domain owners need to test whether common user language maps cleanly to top-level branches, especially at boundaries such as subscriptions versus mandates or payment disputes versus fraud. A router benchmark that samples only clear requests will overstate production reliability.

Completion also needs deterministic checks outside the model. The paper’s runtime makes returns explicit, but a model-generated completion flag can still oscillate or terminate incorrectly. Production systems should enforce maximum local iterations, schema-valid completion states, confirmation gates for sensitive actions, and alerts for repeated branch switching.

Latency must be measured at the workflow level. In the scale benchmark, the hierarchy required four calls instead of one, even though its total measured LLM time at 128 tools was lower in that specific setup. Different providers, network paths, caching policies, and tool runtimes could reverse that result. Prompt savings and wall-clock savings are separate metrics.

The deployment boundary is catalog-specific

Flat routing remains a reasonable choice when the tool set is small, descriptions are distinctive, and one call can inspect the full registry without crowding out user context. Its global visibility can also help when distinctions cannot be expressed cleanly at higher levels.

Hierarchy becomes attractive when total schema exposure threatens context feasibility, nested workflows require reliable return paths, or governance requires local visibility and explicit state transitions. Its value rises with catalog growth and workflow depth, but so does the cost of poor taxonomy.

Most empirical support comes from synthetic catalogs, one model configuration, and route-level evaluation. The paper does not establish multi-model robustness, full business-task completion, or adversarial security performance. Those are deployment tests, not details to assume away.

The architecture therefore changes the investment decision. Teams should not ask only whether a larger context window can hold more tools. They should decide whether the catalog has become large and structured enough to justify local discovery, stack-based control, and the governance machinery that makes hierarchy reliable.

Cognaptus: Automate the Present, Incubate the Future.