Lean

TL;DR for operators A deterministic verifier can tell you whether an artifact satisfies a formal specification. It cannot tell you whether that specification faithfully represents what the user originally asked for. Magenta makes this distinction explicit. The system first converts a natural-language mathematics problem and proposed answer into a Lean statement, then uses a learned judge to check whether that statement preserves the intended problem before attempting a machine-checked proof. When proof verification fails, another judge classifies the failure as mathematical or formal-implementation related and sends the next correction to the corresponding component.1 ...

TL;DR for operators When an AI system produces a formal proof, an evaluation team can make one part of the verdict unusually objective: either the proof is accepted under the allowed rules, or it is not. That removes much of the grader variance found in rubric scoring or LLM judging. AxQM provides 1,019 Lean 4 proof-synthesis tasks over 479 finite-dimensional quantum-mechanics textbook items. It keeps a private reference solution for every task and grades submissions through successful compilation, absence of sorry in the proof or its dependencies, and absence of newly introduced axioms.1 ...

Mathematics has a management problem. That sounds less romantic than saying it has a reasoning problem, but romance is not usually where bottlenecks hide. A proof can be brilliant, a referee can be diligent, and still the verification system can fail for the boring reason that nobody has enough time to check everything line by line. The paper Automatic Textbook Formalization takes that bottleneck seriously and then does something unusually concrete: it reports a multi-agent system that formalized a 500-plus-page graduate algebraic combinatorics textbook into Lean, with all 340 target definitions and theorems proved, in about one week.1 ...

When Less Proves More: The Case for Minimalist AI Theorem Provers Proof is a good place to test AI humility. In ordinary business writing, a model can sound confident, cite familiar patterns, and still be quietly wrong. The error may not surface until the contract is signed, the policy memo is circulated, or the spreadsheet has already acquired the authority of a sacred object. In formal theorem proving, the arrangement is less polite. The model writes code. Lean compiles it. The compiler either accepts the proof or sends it back covered in red ink. ...

Compute is a very convenient alibi. When an AI system fails, the modern reflex is to ask for more of it: more samples, more tokens, more search, more GPUs, more patience from whoever is paying the invoice. This habit is not always wrong. Sometimes the model really does need another attempt. Sometimes the winning answer is hiding in sample number 47. ...

A coder does not trust a program because it sounds plausible. A coder runs it, reads the error message, changes the implementation, tests again, searches the library, asks a colleague, splits the problem, and keeps going until the machine stops complaining. That mundane loop is the interesting part of Numina-Lean-Agent: An Open and General Agentic Reasoning System for Formal Mathematics.1 The headline result is easy to market: with Claude Opus 4.5 as the base model, Numina-Lean-Agent solves all 12 Putnam 2025 problems in Lean, matching the reported perfect score of AxiomProver. Nice. The trophy cabinet sparkles. ...