Scientific Reasoning

TL;DR for operators An internal research assistant may need to discover several technical relationships before it can answer a question. Selecting documents or graph records only once, before reasoning begins, can exclude relationships that become relevant later. Prospector1 demonstrates that this search behavior can be learned by a compact 4B-parameter language model. On 195 semiconductor questions, it achieves 83.9% single-pass accuracy, compared with 87.7% for the strongest evaluated frontier model using the same graph tools. Selecting among 32 sampled episodes with a trained verifier raises Prospector’s score to 91.3%, but adds substantial inference work. ...

TL;DR for operators If two research systems start from the same candidate papers, use the same reranking process, have the same ten-paper writing budget, and use the same GPT-4o writer, should their final citations differ much? In Huang et al.’s study1, they do. When the writer also receives structured views that break papers into claims, premises, reasoning steps, conditions, and provenance, citation F1 rises by 5.3 points on ScholarQA-CS and 5.1 points on ScholarQA-Multi, with both precision and recall improving. ...

TL;DR for operators SciPredict finds that frontier LLMs predict outcomes of recent natural-science experiments with roughly 14-26% accuracy, compared with about 20% for domain experts. That headline can make model performance look surprisingly competitive. It should not be read as evidence that these systems are ready to decide which experiments can safely be skipped. ...

TL;DR for operators SciR is a benchmark for a problem that enterprise AI teams keep trying to flatten into one metric: can a model reason scientifically?1 The more useful question is less flattering and more operational: did the model fail because it could not infer the answer, or because it could not recover the premises from the scientific mess placed in front of it? ...

Brainstorming Is Cheap; Research Judgment Is Not Brainstorming with an LLM is easy. Ask for ten research ideas, wait a few seconds, and receive a confident menu of things that sound just plausible enough to be dangerous. Turn up the temperature and the machine becomes “creative.” Wonderful. We have successfully automated the whiteboard intern. ...

Calculator. That is the boring object hiding inside many “AI reasoning” debates. In technical work, the uncomfortable question is not whether a language model can explain a formula with academic confidence. It is whether the model can still get the answer right after the numbers change, the wording shifts, the unit conversion becomes annoying, and no multiple-choice option politely waves from the corner saying, “Pick me.” ...

Scientific Reasoning Under the Microscope: How PRiSM Stress-Tests the New Generation of Multimodal Models Grades are comforting. A model solves 80% of the benchmark, the leaderboard smiles, the demo team relaxes, and someone in procurement quietly starts asking whether the engineering team still needs that many humans. This is usually the part where reality coughs politely. ...