Reward-Models

Search looks simple until it becomes a workflow. A human analyst can open ten tabs, notice which source contradicts which, remember that one earlier search result changed the meaning of the question, and decide whether the next move should be another search, a calculation, or a final answer. An LLM agent can also open tabs, call tools, browse pages, run code, and produce a final answer. The difference is that the agent often does all of this with the discipline of a caffeinated intern who has been told that “more context” is the same thing as “better memory.” ...

TL;DR for operators The useful lesson is not that vision-language models need longer reasoning traces. They already produce plenty of words. Some of them are even adjacent to thought. The useful lesson is that multimodal systems need feedback that can tell where a reasoning path breaks, not merely whether the final answer looks acceptable. ...

TL;DR for operators Every GUI automation project has a familiar failure mode: the agent gets almost there, makes one bad click, and the training system treats the whole episode as garbage. That is tidy for spreadsheets and absurd for learning. ProgRM addresses that absurdity by replacing final-only success/failure rewards with step-level estimates of task progress.1 Instead of asking only, “Did the agent finish?”, it asks, “How much closer is the agent now than it was one step ago?” The reward is the change in estimated progress. A search that reaches the right article but fails to bookmark it is no longer equivalent to an agent staring at the home screen and scrolling like a caffeinated intern. ...