From ETL to Orchestral Intelligence: The Rise of the Data Agent
TL;DR for operators Most enterprise data work is not blocked by a lack of models. It is blocked by orchestration. A company may already have Spark, Pandas, SQL engines, notebooks, dashboards, semantic layers, data lakes, vector stores, ETL jobs, monitoring tools, and a growing pile of LLM wrappers. The awkward part is deciding which tool should act, in what order, on which data, under which assumptions, and how to recover when the first plan fails. This is the gap the Data Agent paper tries to formalise.1 ...