Verification

TL;DR for operators A robot that opens a door has made a route accessible, but it has not necessarily found the object it was sent to retrieve. In a 3,000-episode simulated benchmark, an agent that maintains interaction-state memory and verifies physical changes achieves 41.63% terminal success, compared with 10.53% for an adapted baseline. Replacing execution-informed state updates with perception-only updates produces a particularly large performance decline in a smaller component test. ...

TL;DR for operators An AI system can save a successful workflow and still spend substantial effort reconstructing how that workflow was originally assembled. Efficient reuse depends on whether the completed procedure is directly available to subsequent planning as a callable operation. FormCraft proposes a capability architecture in which primitive and composed operations share typed interfaces, retain their formation provenance, and can participate in further compatible composition. Verification determines which operations enter a persistent library. ...

TL;DR for operators A robot’s current camera view may show what it is doing without revealing how much of its assigned task it has completed. In a paired evaluation of five frozen progress reward models, supplying the correct active-subtask context reduced mean absolute progress error by 77–82%. The improvement occurred on 93.3% of annotated subtask intervals. These results come from manually supplied context, not autonomous inference.1 ...

TL;DR for operators More AI assessment attempts can increase the chance of resolving a security obligation, but attempt count is not assurance coverage. The operational quantity is the share of distinct assurance outcomes correctly resolved at least once. Jithin VG and Ditto PS develop a theoretical framework for compute-bounded security assurance.1 Its central result is that, under a fixed procedure with conditionally iid repeated attempts, long-run coverage depends on whether some outcomes are unreachable by that procedure. Mean success and pairwise correlation do not reveal that limit. ...

TL;DR for operators An agent that repeatedly uses a procedure and receives high task reward still has not shown that the procedure deserves permanent promotion into its reusable playbook. R² Flow separates three questions that agent systems often blur together: how much a skill participates in successful execution, whether choosing that skill actually improves outcomes relative to alternatives, and whether there is enough independent evidence to authorize a persistent library change. Across eight recursive phases, the paper reports mean held-out OOD score rising from 71.2 to 81.02, with 33 of 37 committed edits improving held-out verified score. By contrast, reward-derived edit labels can raise training reward while lowering that verified score. ...

TL;DR for operators A generic instruction such as “pay more attention” is not a reliable reasoning control. In this study, that prompt barely changes performance for one model, substantially improves another, and makes a third slightly worse. A prompt that instead tells the model which failure points to verify reduces incorrect answers across all three tested model families. ...

TL;DR for operators A tool agent may need to read records, retrieve identifiers, inspect intermediate state, and only then issue the write that completes a task. If the verifier scores only that final write, a seemingly precise per-turn reward can assign learning credit to one visible action while leaving the prerequisite chain unsupervised. ...

TL;DR for operators When one model answer can trigger a consequential downstream step, spending more inference compute before trusting that answer can help—but the form of that spending matters. In the reported experiment, generating several reasoning attempts and aggregating the answer that repeatedly emerged increased verified acceptance from 56.2% to 64.9%. Asking the same model to critique and revise itself produced a smaller increase, from 47.2% to 50.6%. Adding a second model did not improve the reported acceptance rate: 47.4% of outputs survived cross-model verification versus a 48.7% single-model baseline. ...

TL;DR for operators A model that is highly reliable at applying one explicit rule transition is not necessarily reliable at executing an entire procedure built from those transitions. In Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing1, the strongest evaluated model, Gemini 2.5 Pro, achieves 95.6% exact success on one-step next-state generation. At five dependent state transitions, exact success falls to 73.4%. When the model must also choose actions during those five steps, it falls again to 65.3%. ...

TL;DR for operators A model reads a chart, diagram, or photographed math problem and produces the wrong answer. Treating that event as a generic “reasoning failure” can send engineering effort to the wrong component. The model may have misread a number, attached a label to the wrong object, confused a scale or unit, or reasoned incorrectly after extracting the right facts. ...