Cache Me If You Can: Designing Databases for Swarms of AI Agents

A data analyst asks a database a question. An AI agent interrogates it.

That distinction sounds theatrical until the query logs arrive. The human analyst usually knows roughly where to look, asks a small number of targeted questions, waits for answers, adjusts, and eventually presents a result. The agent is less graceful. It checks schemas, samples columns, guesses joins, inspects distinct values, tries partial SQL, abandons it, starts again, validates, retries, and occasionally recruits more agents to repeat the exercise in parallel. It is not being stupid. It is compensating for a missing sense of the underlying data.

That is the useful idea in Supporting Our AI Overlords: Redesigning Data Systems to be Agent-First, a Berkeley-led research vision paper on what the authors call agentic speculation.¹ The paper is not primarily another text-to-SQL story. That would be too small, and frankly too comforting. The argument is that agent workloads are structurally different from the workloads databases were built to serve. If that is right, then better prompting is only the visible aspirin. The headache lives lower in the stack.

Agents speculate because they are powerful and under-grounded

The paper’s central mechanism is simple: LLM agents can reason and call tools, but they usually do not begin with grounded knowledge of the database. They do not know which tables matter, how columns are encoded, where missing values hide, which joins are meaningful, or whether a query failed because the logic is wrong or because “CA” is stored as “California.” Humans learn these things socially, painfully, and slowly. Agents learn them by poking the system.

The authors call this combined process of discovery and solution formulation agentic speculation. It has four properties:

This is the shift that matters. The database is no longer serving a neat stream of independent analytical queries. It is serving a swarm of partially informed attempts to decide what to ask next. Lovely. The intern has become an army.

The evidence is modest, but the shape of the workload is clear

The paper uses two case studies. They are not presented as production-scale proof. Their likely purpose is narrower: to establish that today’s agents already show the workload shape the authors want databases to support.

The first case study uses the BIRD text-to-SQL benchmark, DuckDB as the backend, and two models: GPT-4o-mini and Qwen2.5-Coder-7B-Instruct. The authors test whether more attempts help, either in parallel or sequentially. In the parallel setup, several “field” agents independently attempt the task, and an “agent-in-charge” selects from the candidate solutions. In the sequential setup, a single agent keeps issuing queries until it is satisfied.

The result is not subtle: success improves as the number of requests rises, by 14% to 70% in the case study. That is the first important signal. Agentic speculation can be wasteful, but it is not random thrashing. More attempts can buy accuracy.

The same case study then asks a second question: if agents make many attempts, how much of the work overlaps? The authors examine 50 independent GPT-4o-mini attempts per BIRD problem and compare total subexpressions against unique subexpressions in the resulting query plans. Across sub-plans, the number of distinct sub-plans is often less than 10% to 20% of the total.

That is the database person’s eyebrow-raiser. If many agents repeatedly scan, filter, project, aggregate, or join similar structures, then a normal “each query is independent” execution model leaves money on the floor. The value is not merely caching final answers. It is sharing intermediate work across a cloud of near-duplicate attempts.

The second case study is more realistic and messier. The authors evaluate an agent working across two separate backend databases chosen from PostgreSQL, SQLite, MongoDB, and DuckDB. The tasks require combining information across systems and often using Python, so they cannot be solved as a single neat SQL query. The authors collect 44 sequential traces from OpenAI’s o3 model across 22 tasks, with roughly half of the attempts producing correct answers.

They manually label agent actions into phases: exploring tables and sample data, exploring column statistics, attempting part of the query, and attempting the entire query. The resulting pattern is what one would expect from an analyst, except compressed and multiplied: metadata and samples tend to come first, statistics follow, and partial or complete query attempts appear later. But the phases overlap rather than cleanly replacing one another.

That matters because a database cannot simply classify “early query” as exploration and “late query” as solution. Agent behaviour is more blended. Exploration leaks into formulation; validation appears throughout. Any agent-aware database interface must carry more intent than SQL alone can express.

The paper also tests whether grounding hints make the agent more efficient. Human expert-provided hints are injected into the prompt, such as information about which column contains relevant task information. The average number of SQL queries per trace drops from 12.67 to 10.38, an 18.1% reduction. The biggest relative reduction is in partial query attempts, which fall from 4.28 to 2.71, or 36.6%.

This is not a full benchmark of an implemented agent-first database. It is closer to a controlled demonstration that better grounding can reduce needless probing. Still, the operational implication is sharp: if a backend can supply the right hints at the right time, the agent does less wandering.

The interface problem: SQL cannot say “I am still scouting”

The paper’s proposed architectural response begins with a small but important vocabulary change. Agents should not send only queries. They should send probes.

A probe may contain SQL, but it can also include a natural-language brief: the agent’s goal, the current phase of work, desired accuracy, priority among alternatives, termination conditions, or budget constraints. In other words, the probe tells the database why the request exists.

That extra context changes what the database can do. If the agent is merely exploring, the database may return coarse statistics, samples, approximate results, related tables, or a warning that the request is likely irrelevant. If the agent is formulating a final answer, the database can spend more effort on precision. If the agent submits several possible queries and says only one needs to be completed, the system can choose the cheapest useful path.

SQL is not built to express this kind of intent. LIMIT 10 says less than people pretend it says. It does not explain whether the agent wants examples, schema grounding, confidence, cheap approximation, or a fast reason to abandon the branch.

The authors also argue for semantic operators that go beyond SQL’s conventional matching. Agents may need to ask for tables, columns, or rows semantically related to “electronics,” “employee availability,” or “customer churn,” without knowing where those terms appear. Today, that often means bolting a vector database or metadata service beside the warehouse and hoping the orchestration layer behaves. The paper’s stronger claim is that this semantic grounding should become native to the data system, or at least tightly integrated with it.

For business readers, the lesson is not “replace SQL.” That is the usual performative nonsense. The lesson is that SQL alone is too narrow as an agent-system contract. The next useful interface includes intent, phase, approximation tolerance, and decision context.

The database should talk back before the agent wastes another hour of compute

The paper’s most practical idea is not that agents query databases. It is that databases should answer with more than rows.

The authors describe data systems that proactively provide grounding feedback: related tables, join candidates, why-not explanations for empty results, cost estimates, suggestions to batch related probes, or warnings that a similar query has already been answered. They imagine “sleeper agents” inside the data system that gather such information in parallel and return it as a side-channel.

This is where the misconception about text-to-SQL becomes expensive. If the only goal is better query generation, then the database remains a mute oracle. The agent asks, the database answers, the agent asks again. The paper’s alternative is closer to a data engineer whispering: “That column stores full state names, not abbreviations,” or “You probably want this other table,” or “This scan is costly; sample California first.”

The hint experiment supports this direction, but does not settle the engineering design. Human expert hints reduced SQL actions in the case study. A deployed system would need to generate hints automatically, decide when hints are worth the cost, and avoid leaking information across users or tenants. There is a useful product here, but also a governance trap wearing a nice blazer.

Probe optimisation means satisficing, not worshipping exactness

Traditional query optimisation asks how to execute a query efficiently while preserving exact semantics. Agentic speculation adds a more awkward question: should this probe be executed fully at all?

The paper suggests that an agent-first optimiser should satisfice. It should provide enough information for the agent to make its next decision, not necessarily compute every answer to completion. That might mean pruning irrelevant probes, approximating early exploratory requests, executing one high-value query exactly to reduce future follow-ups, or sharing work across a batch of related probes.

This is not a rejection of accuracy. It is an argument about sequencing. During exploration, an approximate signal can be more valuable than a precise answer to the wrong question. During final formulation, exactness matters more. The optimiser’s job becomes dynamic: decide what to run, how accurately to run it, what to cache, and when extra computation now will reduce total computation later.

That objective fits agent workflows better than conventional throughput metrics. In an agent setting, the relevant unit is not always query latency. It may be time to next useful decision. A database that returns a rough but directionally useful result in two seconds may beat one that returns a perfect answer in twenty seconds, if the agent only needed to choose whether to continue down that branch.

This is also where redundancy becomes commercially interesting. The paper’s BIRD analysis suggests large overlap among sub-plans across parallel attempts. That points toward multi-query optimisation, partial-result caching, approximate query processing, and incremental execution—but adapted for probes with different goals and accuracy needs. The old ingredients exist. The recipe changes because the diner is now a swarm.

Agentic memory is a pseudo-index for meaning, not just a cache for rows

The storage proposal is an agentic memory store: a persistent, queryable semantic cache for grounding information. It may store prior probe results, partial solutions, column encodings, missing-value patterns, date and location granularities, notes about table relevance, or “what worked last time.”

This is not quite a normal index. An index helps locate data efficiently. Agentic memory helps locate meaning efficiently. It reduces repeated discovery by preserving the small bits of context agents keep relearning: which table contains the operational definition of revenue, which column stores territory names, which values are dirty, which join has historically worked, and which prior probe was a dead end.

For enterprises, this is one of the easiest ideas to underestimate. Many organisations already have data catalogues, semantic layers, lineage tools, vector stores, and metadata tables. The paper does not say those disappear. It suggests that agent workloads will make their integration with execution much more important.

The hard part is maintenance. Memory can go stale when schemas change, data distributions shift, or agents write incorrect annotations. Sharing memory across users can also improve efficiency while creating privacy risk. A cache that says “where employee availability is stored” may look harmless until aggregated across enough sensitive workflows. The paper flags this tension rather than solving it.

So the business interpretation should be disciplined: agentic memory is promising where repeated exploration is expensive and data semantics are stable enough to reuse. It is dangerous where access boundaries are weak, metadata quality is poor, or stale context can silently corrupt downstream decisions. A semantic cache is still a cache. Sometimes the thing it preserves is a mistake.

Branched transactions are where agent workloads stop being read-only theatre

Most enterprise agent demos still live in the safer world of retrieval and analysis. The paper looks further: agents that transform and update data.

Here the workload becomes stranger. An agent tasked with rescheduling a delayed flight, reconciling operational records, or proposing database updates may explore many hypothetical worlds. It may fork state, apply a candidate update, test consequences, roll back, and compare alternatives. The authors describe this as requiring support for branched updates and “multi-world isolation.”

The paper notes an observation from Neon: agents created 20 more branches and performed 50 more rollbacks relative to humans. The point is not that every enterprise will see exactly those multiples. The point is that agents treat branching and rollback as a natural part of problem solving. Humans avoid operational branching because it is cognitively and administratively expensive. Agents do it because forking is how they search.

Traditional transactions assume a more linear thread of execution. Agentic speculation wants many near-identical branches that are logically isolated but physically share as much state as possible. The authors compare the needed direction to “MVCC on steroids”: thousands of near-identical snapshots, fast aborts, efficient copy-on-write, and no cross-contamination between worlds.

This is still a research agenda, not a procurement checklist. But it is a useful warning for anyone building agents that will eventually write to production systems. Read-only agent analytics is the training pool. Branched transactional work is the open ocean, with sharks, auditors, and rollback logs.

What leaders can do before vendors rebuild the database

The paper is a systems research vision, so it should not be read as an implementation manual. Still, the business pathway is concrete enough.

First, instrument agent traces. Log not only final SQL, but the whole probing pattern: schema checks, samples, statistics, partial attempts, validation queries, failed branches, repeated joins, and follow-up prompts. Without trace data, “agent cost” remains a mood rather than an engineering target.

Second, classify probes by phase and intent. Even a thin wrapper around existing tools can collect whether the agent is exploring metadata, testing a partial hypothesis, validating a result, or preparing a final answer. That classification is the first step toward better admission control, approximation policy, and caching.

Third, identify redundancy. If agents repeatedly touch the same sub-plans, tables, filters, joins, or metadata questions, build targeted caches. Start boring. Cache column summaries, schema notes, join candidates, sample sketches, and known encoding quirks before fantasising about a grand unified semantic brain. Grand unified semantic brains have a way of becoming expensive filing cabinets.

Fourth, return grounding feedback. Add side-channel hints: related tables, empty-result explanations, cost warnings, and “similar answer already exists” notices. The paper’s hint result suggests this can reduce probing, though production effectiveness will depend on hint quality and task mix.

Fifth, sandbox write operations. Before letting agents modify operational data, invest in cheap branching, strict quotas, fast rollback, provenance, and human approval paths. Agentic writes without branch discipline are not innovation. They are an incident report with a nicer demo video.

A practical scorecard might look like this:

The boundary: this is a research vision, not a warehouse benchmark

The paper’s contribution is conceptual and architectural, backed by illustrative case studies. That is appropriate for the claim it makes, but it limits how aggressively the results should be commercialised.

The 14% to 70% accuracy improvement shows that more attempts can help in the tested text-to-SQL setting. It does not prove that every agent task should be parallelised. More attempts also mean more cost, more contention, and more opportunities for correlated error.

The redundancy result shows large overlap in generated query sub-plans under the tested conditions. It does not prove a universal cache hit-rate for enterprise data agents. Workload structure, schema complexity, model behaviour, and orchestration design will all matter.

The hint experiment shows that useful grounding can reduce query count. It does not prove that an automated sleeper agent can reliably produce those hints without cost, latency, or privacy problems.

The architecture itself—probes, briefs, side-channel feedback, probe optimisation, agentic memory, and branched transactions—is best read as a research map. Some pieces can be approximated today with orchestration layers, catalogues, vector indexes, query logs, and sandbox databases. The deeper version requires database vendors and platform teams to expose agent-aware interfaces natively.

That distinction is important. The paper does not say, “buy a new database because agents are magical.” It says the workload is changing in ways that make old assumptions less comfortable. A refreshingly rude thing for a systems paper to do.

The business value is controlled exploration

The lazy version of the agent story says: give models tools and they will automate analysis. The more useful version says: agents will explore more than humans, faster than humans, and often more redundantly than humans. If unmanaged, that becomes cost, latency, and governance noise. If managed well, it becomes a new optimisation surface.

The core business value is not just cheaper queries. It is controlled exploration: letting agents search broadly enough to improve outcomes while keeping the database from becoming an all-you-can-eat buffet for speculative SQL.

That requires new platform habits. Measure probe sequences, not just final answers. Optimise for next-decision usefulness, not only exact query latency. Cache meaning, not only rows. Let the system talk back. Treat branching and rollback as first-class for write-capable agents. And above all, stop pretending that an AI agent is just a fast analyst with worse keyboard manners.

The paper’s title jokes about supporting our AI overlords. The more sober interpretation is that data systems are about to inherit a new class of user: tireless, uncertain, parallel, and oddly persistent. The winning platforms will not merely tolerate that user. They will shape its behaviour.

Cognaptus: Automate the Present, Incubate the Future.