Devops

Most “AI builds the app” demos fail exactly where production begins: integration, state, and reliability. A new open-source framework from Databricks—app.build—argues the fix isn’t a smarter model but a smarter environment. The paper formalizes Environment Scaffolding (ES): a disciplined, test‑guarded sandbox that constrains agent actions, validates every step, and treats the LLM as a component—not the system. The headline result: once viability gates are passed, quality is consistently high—and you can get far with open‑weights models when the environment does the heavy lifting. ...

In the race to build autonomous software engineers, large language model (LLM) agents like Devin and Copilot Chat are lauded for fixing bugs, writing code, and even completing tasks from GitHub issues. But what happens when the code doesn’t even run? That’s the uncomfortable gap SetupBench aims to measure—and the results are sobering. SetupBench introduces a 93-task benchmark evaluating a foundational but under-tested skill: bootstrapping a development environment from scratch. Unlike prior benchmarks that hand agents a fully pre-configured Docker container, SetupBench drops them into a barebones Linux sandbox and challenges them to install dependencies, initialize databases, configure background services, and resolve real-world version conflicts. It sounds simple. It isn’t. ...