The Right Number Can Still Answer the Wrong Question
TL;DR for operators A financial agent can retrieve the correct filing, extract a defensible figure, ask a relevant follow-up question—and still return the wrong answer for what the user meant. The clearest evidence is the gap between knowing the interpretation and having to recover it. When evaluated models are supplied with the disambiguating interpretation, their accuracy rises to roughly 90-95%. Under full free interaction, where the agent must decide what to clarify and use the answer correctly, the best result is 28.9%. ...