How Many Passes Does an AI Decision Need?

TL;DR for operators

An automated decision service may benefit from reconsidering a question before selecting an answer. But additional computation has a cost, and a more confident answer is not necessarily more trustworthy.

SanSi demonstrates a configurable approach: repeatedly process the same input through shared transformer layers and produce option probabilities after each pass. Across its evaluation suite, accuracy improves substantially within three loops, with much smaller gains thereafter. Eight loops achieve 72.0% accuracy but cost 7.7 times the GPU computation of a one-loop Ouro pass. Probability calibration also deteriorates after the third loop.

For operators building classification, routing, or document-review services, the opportunity is to treat inference depth as an operational choice, while measuring accuracy, latency, confidence reliability, and missing-evidence detection separately. The reported adaptive-stopping savings remain largely simulation-based, and model confidence does not replace authorization to act.

The answer improves before the computation ends

Consider an automated service routing customer requests. Some requests contain enough information for an immediate decision. Others require connecting details scattered across a message, a policy, and an attached document.

The service could choose an option immediately or spend additional computation reconsidering the available evidence. The difficulty is knowing when that extra work has earned its cost.

In the experiments reported by Gan, Lee, and Kang, a model’s accuracy rose from 58.4% after one computational pass to 70.4% after three. Five additional passes raised it to 72.0%. Meanwhile, its expected calibration error—a measure of how closely predicted confidence agrees with observed correctness—worsened from 0.082 at the third pass to 0.093 at the eighth.¹

The operational question is therefore more specific than whether longer reasoning helps. A decision service must determine which intermediate answer to use, how much processing to purchase, and whether its reported probabilities are suitable for the intended action.

Repeated computation without generated reasoning text

The researchers introduce SanSi, a typed decision model. Given a state, a question, and explicitly declared options, it returns a probability distribution over those options instead of writing a free-form response.

Its architecture builds on the Ouro family of language models, pretrained to repeatedly apply the same transformer layers. Each iteration processes the hidden state from the preceding iteration. The backbone parameters are reused, so computational depth increases without creating another complete model.

SanSi freezes its pretrained backbone and trains shared LoRA adapters, together with small loop-specific output components. Every loop produces its own option probabilities.

Crucially, the model is trained to make these intermediate predictions meaningful. Its objective combines cross-entropy and the Brier score at every loop, rewarding both selection of the correct option and distributional accuracy.

A training ablation demonstrates why this matters. With supervision only at the final loop, first-loop accuracy falls to 35.6%, compared with 58.4% under the full per-loop objective. Final-loop accuracy also declines, from 72.0% to 70.9% (Section 7, Table 3).

This is what the authors call System 1.5 thinking: repeated internal computation between a single-pass prediction and explicit, generated reasoning. It does not produce a shortened chain of thought, and the experiments do not establish a unique internal reasoning algorithm.

Most of the accuracy arrives in three loops

The principal evaluation contains 10,027 decisions from 59 sources, spanning classification, multi-step reasoning, uncertain evidence, long documents, sentence pairs, and knowledge questions. The authors compare models trained on the same decision data, reporting main results across three training seeds.

Source: Section 4, Table 2, and Section 5.2. GPU cost is relative to one Ouro-1.4B pass on an RTX A6000, not a universal production-latency estimate.

Against the same-shape SmolLM2 control, eight-loop SanSi gains an estimated 13.5 percentage points, with a 95% bootstrap interval of 12.5–14.5 points.

The improvement is particularly pronounced on knowledge, long-document, and multi-step reasoning decisions. Ordinary classification sees a much smaller gain. This suggests that the value of repeated computation depends on the type of information processing required.

The reasoning-depth experiments provide a more targeted test. On synthetic liar-chain tasks at unseen depths 9–16, eight-loop SanSi achieves 72.6% accuracy against 50.0% for Qwen3.5-4B. On unseen object-swap depths, the respective figures are 63.7% and 34.2% (Appendix G.2, Table 31).

Those experiments deliberately manipulate sequential dependency depth, strengthening the case that recurrence can support deeper computation. They are not evidence of general reasoning superiority across unrestricted business problems.

A stable answer can still be wrong

SanSi’s intermediate predictions reveal that continued computation is not uniformly corrective.

Among answerable decisions, 56.7% never change their initial answer, and 89.1% have settled by loop four. Between the first and eighth loops, 21.0% of decisions are corrected, but 7.3% move from correct to incorrect.

A stable prediction is therefore informative about processing behavior, not proof of correctness.

This motivates adaptive stopping. The researchers evaluate several stopping rules using recorded loop predictions. Ouro’s pretrained exit gate maintains approximately 72.0% accuracy while averaging 4.5 loops instead of eight, implying roughly 44% less recurrent computation under per-item stopping assumptions (Section 5.2, Tables 22–24).

For an operator, that suggests a possible inference policy: let straightforward requests finish early and allocate more computation where additional processing is likely to help.

The qualification is substantial. Static batching may prevent the system from realizing the simulated savings, and a full dynamic-exit production implementation was not established by the experiments.

Accuracy, confidence, and evidence require different checks

Repeated computation can improve answer selection while degrading probability reliability.

SanSi’s mean confidence increases on both correct and incorrect decisions. Expected calibration error improves from 0.105 at loop one to 0.082 at loop three, then rises to 0.093 at loop eight.

The model is becoming more accurate overall, but its probabilities are not becoming consistently better calibrated.

The researchers test a simple remedy: averaging option probabilities across all eight loops. Accuracy remains close to the final-loop result, moving from 72.0% to 71.8%, while expected calibration error falls from 0.093 to 0.044 (Section 7, Table 41).

This calibration improvement requires no additional inference beyond computing the eight loops. It does not, however, deliver the savings of stopping early. An early-exit system would need its own probability aggregation and calibration evaluation.

Missing-evidence detection follows another trajectory. Evidence AUROC rises from 0.835 to 0.935 across eight loops, while the hard-answer rate on 382 unanswerable decisions declines from 27.1% to 17.5%.

These results matter for document-based workflows where the required supporting information may be absent. Still, SanSi does not produce an explicit abstention option: unanswerable cases receive uniform training targets, and the reported evaluation applies a confidence threshold to judge whether the model withheld a hard answer.

For Cognaptus, the implementation implication is to separate three tests: whether the selected option is correct, whether its probability is calibrated, and whether required evidence exists. Improving one does not establish the others.

Where the operational economics remain uncertain

Parameter efficiency should not be confused with computational efficiency.

Eight-loop SanSi has a 1.4B-parameter backbone, but its measured GPU time is approximately 3.3 times that of Qwen3.5-4B. Its lower reported peak inference memory is attractive for some workloads, although the memory advantage depends on how states and intermediate representations are reused.

The architecture also relies on loop-aware pretraining. Retrofitting recurrence onto a conventional backbone does not reproduce its gains, and extending inference beyond the trained loop budget can reduce accuracy. These ablations limit the claim that arbitrary models can simply be run repeatedly.

The authors additionally study SanSi as a verifier for reinforcement learning. On 2WikiMultiHopQA, a SmolLM2-1.7B generator trained using loop-eight SanSi probabilities as its sole GRPO reward improves externally graded correctness from 37.0% to 52.1%. Loop-one verifier rewards provide no statistically clear correctness improvement and reduce token F1 (Section 8, Appendix I.3).

This is a promising exploratory extension: better intermediate evaluation can improve the reward signal used to train another model. But it concerns one generator and one question-answering dataset, with correctness graded by an external language model. It does not establish general-purpose reward reliability.

For business adoption, a practical assessment should use representative requests and compare inference depths on end-to-end latency, GPU utilization, accuracy by decision category, probability calibration, missing-evidence behavior, and the cost of wrong actions. The English-only evaluation, possible public-data pretraining exposure, and incomplete release of some experimental code also constrain external generalization.

The operational value is control over when to stop

SanSi demonstrates that additional sequential computation can improve typed decisions without expanding the backbone’s parameter count or generating reasoning text. Its strongest practical insight is that the best inference budget is not necessarily the maximum available budget.

Cognaptus sees an opportunity for configurable decision services: inexpensive early predictions for routine work, additional processing for difficult cases, calibrated confidence for escalation, and explicit deferral when evidence is insufficient. These are design implications to test, not production outcomes established by the paper.

Finally, a probability distribution may guide a workflow, but it cannot authorize execution. Policy constraints, data validation, and permissions must remain independently enforced, especially when a decision triggers a financial, contractual, or customer-facing action.

The relevant objective is not to make every decision think longer. It is to determine when another pass changes the decision enough to justify its cost—and when the system should stop without acting.

Cognaptus: Automate the Present, Incubate the Future.