When More Geometry Makes the Answer Worse

TL;DR for operators

Adding structured information to an AI reasoning system does not necessarily improve its answers. In one geometry experiment, explicit relationships corrected 28 mistakes but introduced 13 new ones. The problem was not simply missing information: irrelevant constraints and ambiguous references could also mislead the solver.

GeoReform addresses this by optimizing the instructions that determine which geometric facts are extracted, how they are identified, and how they are presented. On Geometry3K, this approach improves accuracy by 4.0–17.5 percentage points over an initial fixed policy, without changing the downstream reasoning model.

For developers of diagram-based AI systems, the implication is worth testing: improve the information supplied to an existing model before assuming a larger model is necessary. The demonstrated benefit remains limited to plane geometry benchmarks, and the study does not establish lower deployment costs.

When adding correct-looking facts introduces new errors

An AI system solving a geometry problem must interpret a diagram and reason about its relationships. A common approach is to extract explicit statements about angles, lines, lengths, and shapes, then provide those statements alongside the original problem.

That additional structure should make relevant relationships easier to use. Yet the results are less predictable than the design suggests.

In a diagnostic involving 200 Geometry3K problems, adding explicit geometric structure to Qwen3VL-8B corrected 28 previously incorrect answers while causing 13 previously correct answers to become incorrect.

The same intervention helped some problems and harmed others.

The authors identify two recurring causes. First, redundant or weakly relevant relationships can distract the solver from constraints needed for the answer. Second, ambiguous geometric references can cause a valid-looking relationship to be applied to the wrong line, angle, or figure.

This makes the quality of the intermediate representation a separate engineering problem. Extracting more facts may increase coverage without improving the solver’s ability to use them.

GeoReform optimizes the representation policy, not the reasoning model

In GeoReform: Reflective Formalization Evolution for Multimodal Geometry Problem Solving, Jialu Wang and coauthors¹ propose improving the instructions that govern geometric formalization rather than modifying the model that ultimately solves the problem.

A geometric formalization translates the diagram and question into structured statements. For example, Parallel(Line(A,B),Line(C,D)) identifies two parallel lines, while Equals(Angle(A,B,C),54) specifies an angle measurement.

Producing such statements requires decisions about what to include, how to identify each entity, which relationships belong together, and how to express the target quantity. GeoReform calls the instructions governing these decisions a formalization policy.

The framework improves that policy through a failure-driven process:

1. Run the complete pipeline. A structure generator interprets the diagram and question under the current policy. A downstream model receives the resulting geometric statements alongside the original inputs.
2. Diagnose unsuccessful solutions. Incorrect answers are examined with their generated formalizations, reasoning traces, and reference answers to identify representation defects.
3. Propose policy revisions. A reflection model suggests reusable changes, such as clearer angle grounding, stronger target identification, or removal of distracting relationships.
4. Test before accepting. Candidate policies must improve performance on a sampled training minibatch before being considered for selection against validation performance.

The reported optimization uses training minibatches of 32 problems, up to 16 failed cases per reflection input, and at most 20 update iterations.

Crucially, the structure generator, downstream reasoning model, and reasoning prompt remain fixed during policy evolution. GPT-5 generates the geometric representations, but the optimization changes the instructions governing those representations rather than fine-tuning the solver.

The resulting policy is reusable. It is selected offline and subsequently applied to new problems; the reflection process does not receive the correct answer for each test problem.

The appendix’s selected policy lineage illustrates the kinds of changes involved. Earlier revisions strengthen grounding of geometric entities, angles, lines, and targets. Later revisions introduce relevant properties of named figures more selectively.

These observations provide a plausible explanation for the improvement, although the paper does not independently quantify the effect of each policy change.

Better formalization improves accuracy across nine model configurations

The central evaluation compares GeoReform with its initial formalization policy on Geometry3K, holding the downstream reasoning setup fixed.

The benchmark distinguishes Choice accuracy, where candidate answers are provided, from Completion accuracy, where the model generates an answer without options.

Selected results from Table 1 show the magnitude of the improvement:

Source: Table 1, Geometry3K. Gains are absolute percentage-point differences relative to the initial formalization policy.

For Qwen3VL-2B, the unaugmented model achieves 33.5% Choice accuracy and 29.0% Completion accuracy. The initial policy already improves those results to 42.0% and 37.0%. GeoReform raises them further to 56.0% and 53.5%.

Keeping these baselines separate matters. The principal contribution is not merely adding structured facts; it is improving the policy that produces them.

Across all nine tested model configurations and both answer settings, GeoReform outperforms the initial policy in every reported comparison. Improvements range from 4.0 to 17.5 percentage points.

The gains are generally larger for smaller models, but they do not decline monotonically with parameter size. Larger models also benefit, as demonstrated by Qwen3VL-32B.

A separate comparison with InterGPS and AutoGPS reinforces the finding. Under a common InternVL3.5 downstream reasoning setup, GeoReform exceeds the strongest competing formalization configuration by 10.0–15.5 percentage points in Choice and 7.5–21.0 points in Completion (Table 2).

That comparison establishes a system-level performance advantage under the tested protocol. It does not isolate which individual representation choices account for the difference between methods.

The additional benchmarks test structure injection, not policy evolution

GeoReform also reports experiments on GeoQA and PGPS9K, but these address a narrower question: whether providing explicit geometric structure improves reasoning.

On GeoQA, GPT-5 extracts representations because official formal annotations are unavailable. Six of eight reported configurations improve. However, Qwen3VL-8B and Qwen3VL-32B Choice accuracy decline by 1.0 and 2.0 percentage points, respectively.

On PGPS9K, the experiments instead use ground-truth geometric annotations. All four evaluated Qwen3VL scales improve in Choice accuracy, with gains ranging from 5.5 to 21.5 percentage points (Table 3).

These findings suggest that reliable explicit structure can be valuable, including when much of the necessary information already appears in the question. They also show that benefits depend on the representation and evaluation setting.

Neither experiment independently demonstrates GeoReform’s full policy-evolution procedure on those datasets. The direct evidence for reflective evolution remains concentrated on Geometry3K.

The paper’s qualitative analysis offers another clue about the mechanism. On rhombus and parallelogram problems, connecting local diagram marks to broader shape properties appears particularly beneficial. But without independently isolating these representation changes or reporting the relevant per-type sample sizes, the cases remain explanatory evidence rather than definitive mechanism tests.

For AI engineering teams, test the representation before enlarging the solver

Cognaptus interprets these findings as a reason to treat the intermediate representation as an independently optimizable part of a multimodal workflow.

Consider a team building a system that answers questions about geometric diagrams. If the current solver makes mistakes because relevant constraints are missing, ambiguous, or poorly organized, replacing it with a larger model may not address the underlying input problem.

A bounded evaluation would compare a fixed extraction policy with a failure-evolved policy while holding the downstream model and reasoning prompt constant. The team should measure not only total accuracy, but also which previously incorrect answers become correct and which previously correct answers regress.

This approach is most defensible when the team has representative labeled problems, inspectable intermediate representations, and enough failed reasoning traces to diagnose recurring extraction defects.

The same evaluation design could be explored in technical drawing interpretation, document understanding, or other multi-stage AI systems. However, those are engineering hypotheses extrapolated from geometry, not applications validated by this paper.

There is also a resource-allocation question. Better formalization may allow a smaller solver to achieve an acceptable accuracy target, but that possibility must be assessed against the additional structure-generation cost. The benchmark does not demonstrate an overall economic advantage over using a larger solver directly.

Better representation does not guarantee reliable or cheaper reasoning

Three limitations affect deployment decisions.

Visual grounding remains fallible. A refined policy cannot guarantee that the structure generator correctly recognizes every mark, segment, or angle. Incorrectly extracted facts can still propagate into downstream reasoning. Independent consistency checks may therefore be necessary for consequential applications, although the paper does not validate such checks as part of GeoReform.

Generalization remains uncertain. The primary policy-evolution experiments concern plane geometry. Transfer to unfamiliar diagram conventions, solid geometry, engineering drawings, or other visual reasoning domains has not been established. The study also lacks a comprehensive repeated-run statistical significance analysis.

Production economics are unmeasured. GPT-5 generates the formalizations, and the authors do not provide comprehensive comparisons of inference latency, token consumption, optimization expense, or operating cost. Equivalent gains using cheaper structure generators have not been demonstrated.

Together, these limitations prevent a direct conclusion that reflective formalization evolution is a more economical or universally reliable architecture.

The next optimization target may be the information interface

GeoReform demonstrates that meaningful accuracy improvements can come from changing how information reaches a reasoning model, without changing the model itself.

Its strongest result is the controlled improvement over a fixed initial policy across the tested Geometry3K configurations. Its broader contribution is an engineering method: use downstream failures to diagnose and improve reusable upstream representation instructions.

For teams already deploying multimodal reasoning pipelines, that suggests a concrete next experiment. Before paying for a larger solver, determine whether the existing one is being supplied with the right constraints, grounded to the right entities, in a form it can reliably use.

The decision should ultimately rest on held-out accuracy, introduced errors, reliability, latency, and total cost—not on the amount of structured information extracted.

Cognaptus: Automate the Present, Incubate the Future.