Evaluation Metrics

TL;DR for operators A CAD assistant that writes code once and hopes for the best is not an engineering workflow. It is a raffle with syntax highlighting. IterCAD is interesting because it treats CAD generation and editing as an iterative operating loop: read the drawing, generate CadQuery code, execute it in a sandbox, inspect compiler and geometric feedback, revise, and stop only when the model has evidence that the shape is right.1 The paper’s practical contribution is not “AI can design parts now.” That would be the usual confetti cannon, and mercifully not the correct lesson. The better lesson is that useful CAD automation needs closed-loop verification, localized visual grounding, and evaluation metrics that count failures instead of quietly hiding them in the basement. ...

AI teams like committees. Not human committees, of course. Those are unfashionable. We now prefer committees made of agents: one agent plans, one verifies, one critiques, one searches, one writes code, one supervises the others, and somewhere in the corner a “coordinator” burns tokens making everyone feel aligned. This architecture is not stupid. Multi-agent systems solve real problems: they divide labor, preserve specialized expertise, and make complicated workflows easier to inspect. But they also bring the usual committee tax: coordination overhead, fragmented context, brittle phase ordering, and the faint smell of process worship. ...