Cover image

The Right Number Can Still Answer the Wrong Question

TL;DR for operators A financial agent can retrieve the correct filing, extract a defensible figure, ask a relevant follow-up question—and still return the wrong answer for what the user meant. The clearest evidence is the gap between knowing the interpretation and having to recover it. When evaluated models are supplied with the disambiguating interpretation, their accuracy rises to roughly 90-95%. Under full free interaction, where the agent must decide what to clarify and use the answer correctly, the best result is 28.9%. ...

October 13, 2026 · 7 min · Zelina
Cover image

Mind the Earnings Gap: Why LLMs Still Flunk Financial Decision-Making

TL;DR for operators A financial AI system does not fail only when it invents a company, misreads a filing, or forgets what EBITDA means. Those are the obvious failures. FinanceBench is more useful because it exposes the quieter failure mode: the model has access to the document, produces a coherent answer, and still gets the financial question wrong.1 ...

July 28, 2025 · 14 min · Zelina