LLM Cascades

TL;DR for operators Most model-selection dashboards still ask the wrong question. They ask which LLM gives the best accuracy for the lowest inference cost. Zellinger and Thomson’s paper asks a more operationally honest one: how much does a wrong answer, a slow answer, or no answer cost in this specific workflow?1 The paper’s useful move is to convert competing performance metrics into a single expected dollar reward. Inference cost stays in dollars. Latency gets priced in dollars per second or minute. Errors get priced by their business consequence. Abstention gets priced by the cost of failing to answer or escalating to a human. Once everything is in the same unit, the “best model” is no longer the one that looks attractive on a Pareto plot. It is the model with the highest expected reward under the actual economics of the task. ...

TL;DR for operators Month-end close is not where small firms discover their love of manual labour. It is where invoices arrive half-labelled, clients reply with attachments named final_final_real.xlsx, and a senior accountant spends expensive hours doing work that is intellectually closer to sorting laundry than advising a business. The practical AI opportunity for small accounting and professional service firms is not “give everyone a chatbot and hope the profession becomes futuristic by Friday.” The better architecture is a cost-aware, privacy-first workflow: classify the task, remove or mask sensitive data where possible, retrieve the right firm knowledge, route the easy work to cheap or local tools, escalate uncertain cases to stronger models, and keep humans in charge of outputs that affect filings, financial statements, tax positions, or client advice. ...