Data Agents

TL;DR for operators A data agent can find the right evidence, perform much of the required analysis, and still fail the task by returning the wrong table. In an audit of 136 failed runs from the strongest tested backbone, 71 failures—52.2%—were attributed primarily to turning the agent’s internal result into the requested output. Sixty of those involved submitting extra or missing columns. Only three failures were attributed to selecting the wrong evidence source. ...

A dashboard fails. The sales team says the numbers changed overnight. The data engineer checks the pipeline. The analyst checks the SQL. The BI vendor says its “agent” can help. The executive hears “agent” and imagines a small autonomous data scientist quietly fixing the mess before breakfast. Usually, no. Usually it is a chatbot with access to SQL, a tool wrapper with better manners, or a workflow assistant that still depends on human supervision at the awkward parts. Useful, yes. Autonomous, no. The distinction is not academic hair-splitting; it determines who owns the error when the agent rewrites a query, changes a pipeline, or confidently explains a metric built on dirty data. ...

TL;DR for operators Most enterprise data work is not blocked by a lack of models. It is blocked by orchestration. A company may already have Spark, Pandas, SQL engines, notebooks, dashboards, semantic layers, data lakes, vector stores, ETL jobs, monitoring tools, and a growing pile of LLM wrappers. The awkward part is deciding which tool should act, in what order, on which data, under which assumptions, and how to recover when the first plan fails. This is the gap the Data Agent paper tries to formalise.1 ...