Evaluation-Metrics

TL;DR for operators A model-risk team estimating failures too rare for ordinary sampling has to decide which kind of error it can least afford. The benchmark shows that this choice can reverse which estimator looks best. :contentReference[oaicite:0]{index=0} Under approximately symmetric penalties, GA-AMLS has much lower average SPB loss than QLD for the 1-layer and 4-layer models. When the loss instead makes underestimation much more costly, QLD sharply outperforms GA-AMLS—even though QLD systematically overestimates and GA-AMLS has lower bias. ...

TL;DR for operators A CAD assistant that writes code once and hopes for the best is not an engineering workflow. It is a raffle with syntax highlighting. IterCAD is interesting because it treats CAD generation and editing as an iterative operating loop: read the drawing, generate CadQuery code, execute it in a sandbox, inspect compiler and geometric feedback, revise, and stop only when the model has evidence that the shape is right.1 The paper’s practical contribution is not “AI can design parts now.” That would be the usual confetti cannon, and mercifully not the correct lesson. The better lesson is that useful CAD automation needs closed-loop verification, localized visual grounding, and evaluation metrics that count failures instead of quietly hiding them in the basement. ...

AI teams like committees. Not human committees, of course. Those are unfashionable. We now prefer committees made of agents: one agent plans, one verifies, one critiques, one searches, one writes code, one supervises the others, and somewhere in the corner a “coordinator” burns tokens making everyone feel aligned. This architecture is not stupid. Multi-agent systems solve real problems: they divide labor, preserve specialized expertise, and make complicated workflows easier to inspect. But they also bring the usual committee tax: coordination overhead, fragmented context, brittle phase ordering, and the faint smell of process worship. ...