The Two Minds of Finance: Testing LLMs for Divergence and Discipline

TL;DR for operators

Finance teams do not ask AI systems to do one kind of thinking. They ask them to imagine plausible futures, extract investable implications, choose between similar explanations, and avoid being seduced by the prettiest narrative. Those are not the same task. A model can be fluent, plausible, and still strategically dull. Finance has a long tradition of rewarding that, but we do not need to automate the habit.

The paper behind ConDiFi makes this split measurable.¹ It evaluates 14 LLMs on two finance-specific tracks: 607 open-ended macro-financial scenario prompts for divergent thinking, and 990 adversarial multi-hop MCQs for convergent thinking. The divergent track asks models to generate branching future timelines. The convergent track asks them to choose the one logically valid timeline among carefully designed distractors.

The rankings diverge. On the open-ended scenario track, Cohere Command A and DeepSeek-R1 lead with overall scores of 8.04 and 8.03 respectively, ahead of Phi-4, o1, and GPT-4o. On the refined convergent MCQ track, Llama4 Maverick ranks first at 72.32%, followed by o1 at 71.41%, Llama4 Scout at 71.01%, and a tie between Llama3 70B and DeepSeek-R1 at 70.10%. No model reaches the paper’s “excellent” threshold above 78% on the hardest refined set.

For operators, the useful conclusion is simple: do not buy “the best model for finance.” Buy, route, and test for the kind of reasoning you actually need. Scenario planning needs novelty, causal branching, and actionable hypotheses. Investment committee support needs disciplined selection, temporal coherence, and resistance to optimistic distractors. Governance needs to know which failure mode is being invited into the room.

ConDiFi is not a trading system, a procurement oracle, or proof that any one model should be trusted with capital. It is better read as an evaluation grammar: a way to separate two cognitive jobs that vendor demos usually flatten into one glossy paragraph.

Finance has two jobs, and most benchmarks test only one

The familiar benchmark question is: did the model get the answer right?

That question is useful. It is also incomplete. In financial work, the answer is often not sitting there like a missing number in a spreadsheet. Analysts have to imagine what might happen next, identify which pathways matter, map second-order consequences, and then converge on a decision under constraints. The market does not hand out answer keys. It hands out ambiguity, then charges fees for every mistake.

ConDiFi is built around that distinction. It separates reasoning into two modes:

This comparison matters because finance AI often fails politely. It does not necessarily hallucinate a spectacularly false number. It may instead produce a beautifully formatted explanation that is too generic to matter, too optimistic to be safe, or too shallow to survive contact with a real investment committee.

ConDiFi’s contribution is not merely that it adds another benchmark to the growing benchmark landfill. Its contribution is that it asks whether a model is good at the kind of thinking the task demands. That is a more annoying question, which is usually a sign that it is the better one.

What ConDiFi actually tests

The benchmark has two tracks.

The divergent track uses 607 scenarios based on real events dated 1 May 2025 or later. Each scenario is a medium-length summary of economics, financial, geopolitical, or political developments relevant to investors. The model is asked to generate a branching timeline of how the situation could evolve.

Those generated timelines are scored on five dimensions: plausibility, novelty, elaboration, actionability, and richness. Four of those are judged using GPT-4o as an LLM-as-judge. Richness is computed structurally by parsing the timeline as a directed tree and measuring branching, depth, mean path length, and number of leaf paths. In other words, the paper tries to distinguish “long answer” from “actually explored multiple futures.” A welcome distinction; verbosity has been impersonating intelligence for a while.

The convergent track uses 990 MCQs. Each question gives a scenario about a NYSE-listed company, followed by four possible event timelines. One option is correct because it satisfies factor alignment, temporal coherence, and logical entailment. The distractors are designed to be confusable, not cartoonishly wrong.

The paper uses six adversarial generation patterns for the convergent questions:

The two-track design is the point. A model that can generate three plausible macro branches may still choose the wrong one when asked to discriminate under tight constraints. Conversely, a model that selects the right MCQ may not be imaginative enough to produce useful scenario trees. Finance needs both. The benchmark shows they do not arrive in the same package by default.

The divergent track rewards useful imagination, not just eloquence

On divergent thinking, Cohere Command A and DeepSeek-R1 dominate the overall ranking. Cohere Command A scores 8.04 overall; DeepSeek-R1 scores 8.03. The difference is tiny, so the sensible interpretation is not “one true champion.” The sensible interpretation is that both models sit in the top cluster for financial foresight generation.

The more interesting pattern is how they get there.

GPT-4o is the revealing comparison. It is plausible. It is fluent. It scores 8.12 on plausibility. But it lags on novelty and actionability. That is exactly the trap business users fall into when they treat analyst-style prose as analyst-grade reasoning. A polished memo can still fail to produce non-obvious, decision-relevant futures. It merely fails in a suit.

The dimension-level statistics reinforce the same point. Across the divergent evaluations, plausibility has a mean score of 7.81 and a relatively tight standard deviation of 1.03. Actionability has a lower mean of 6.24 and a much wider standard deviation of 1.89. Elaboration also varies meaningfully, with a standard deviation of 1.56. In plain English: most models can avoid sounding absurd, but they differ sharply in whether they can turn a scenario into something an investor could actually use.

That distinction matters operationally. If a model is being used to support strategic foresight, plausibility should be treated as a floor, not a differentiator. A plausible but obvious model will give you the consensus with nicer punctuation. The premium capability is structured, decision-relevant divergence: second-order effects, named actors, plausible timing, policy levers, sector exposure, and concrete triggers.

The paper’s manual review adds another useful wrinkle. Models looked more original when the prompt itself contained strong factual structure, such as industrial or supply-chain developments. In more anecdotal or emotionally framed prompts, even strong models fell back on clichés. This is not a small detail. It means “AI creativity” may depend heavily on how much causal scaffolding the input already provides. Apparently, models are more imaginative when the world has already done part of the thinking for them. Shocking, yes.

The convergent track punishes optimism and loose causal reading

The convergent task is a different animal. Here, the model must choose the correct timeline from adversarially similar options. The metric is Convergent Correctness Score, or CCS: the share of answers exactly matching the ground truth.

The paper reports three versions of the convergent dataset: original, refinement 1, and refinement 2. The refinements are not an ablation of model design. They are a difficulty escalation in the benchmark construction pipeline. After the first refinement, average CCS falls from 85.45% to 83.31%. After the second, it drops sharply to 65.61%.

That second refined set is where the comparison becomes useful.

The ranking is not the same as the divergent track. Llama4 Maverick leads the refined convergent benchmark, despite sitting near the bottom half on divergent overall score. Cohere Command A, the top divergent model, is mid-pack on convergent correctness. DeepSeek-R1 is one of the few models that remains strong on both sides, though it does not lead the convergent track.

The benchmark also reveals that the task remains hard. The paper defines poor convergent ability as 55% or less, good ability as 55% to 78%, and excellent ability as above 78%. No evaluated model exceeds 78% on refinement 2. Even the best model, Llama4 Maverick, reaches 72.32%. That should cool any temptation to declare financial reasoning solved. It is not solved. It is, at best, becoming measurable.

The error analysis is particularly useful because it examines mistakes by the best-performing convergent model. The dominant error categories are not exotic. They are depressingly familiar:

The first two are the ones operators should care about most. The model often did not fail because it lacked a grand theory of finance. It failed because it missed the emphasis, weighted the wrong driver, or let a positive milestone dominate a negative constraint. That is close to how human committees fail too, except the humans usually call it “judgment.”

The optimism bias is also worth isolating. In finance, a model that prefers clean positive timelines can look helpful in growth narratives and dangerous in risk work. If the task is credit review, regulatory exposure, supply-chain stress, or downside scenario planning, optimism is not a personality trait. It is a control failure.

The split rankings are the business lesson

The temptation is to ask which model won. That is the wrong question.

The better question is: won what?

This is where ConDiFi becomes useful for procurement and governance. A financial institution should not evaluate models only on broad reasoning scores, generic MMLU-style metrics, or vendor-provided examples. It should build a task portfolio.

For open-ended research, the evaluation should ask: does the model generate branches that are novel but economically plausible? Does it move beyond first-order effects? Does it identify tradeable or hedgeable implications? Does it merely elaborate, or does it actually widen the future state space?

For constrained decision support, the evaluation should ask: can the model select the correct sequence when several options are plausible? Does it overweight good news? Does it notice procedural constraints? Does it preserve the logic of the scenario even when the scenario is intentionally counterintuitive?

This is also a routing argument. A production finance agent does not need one model to do everything. It may use one model to generate divergent scenario trees, another to critique causal coherence, a third to extract decision triggers, and a fourth to perform adversarial review. The article version of that idea sounds elegant. The operational version requires logging, scorecards, escalation thresholds, and someone willing to tell the model “no.” Less glamorous, more useful.

The fingerprint analysis is interesting, but exploratory

ConDiFi goes beyond raw rankings by studying how evaluation dimensions correlate within each model. The paper uses pairwise correlations, Frobenius distance between correlation matrices, and PCA to compare “reasoning fingerprints.”

This part should be read as exploratory diagnostic analysis, not as a second main proof. It helps ask whether models behave similarly, where they differ, and whether their strengths might be complementary in ensemble workflows.

The correlation analysis surfaces three patterns:

The last point is the most important. A model can write a sophisticated single-path story and still fail divergent thinking. Scenario planning is not just “tell me more.” It is “show me materially different ways the future could unfold, and what would make each path more or less likely.”

The Frobenius distance analysis tries to identify similar and dissimilar model behaviours. For example, the paper reports that o4-mini and o1-mini have close correlation behaviour, while DeepSeek-R1 emerges as an outlier in several comparisons. PCA then places models into rough behavioural groupings, with the authors cautiously associating components with possible training or alignment differences.

The word “cautiously” matters. These diagnostics are useful for model selection and ensemble design, but they do not prove why the models behave differently. Training-data access is limited. Proprietary model details are incomplete. PCA labels such as “alignment with human preference” or “reasoning style” are interpretive. They are hypotheses, not autopsies.

For operators, the fingerprint idea is still valuable. If two models make errors in the same way, an ensemble may simply give you redundant confidence. If their reasoning fingerprints differ, one model may catch what another misses. That is the beginning of useful redundancy.

What the paper directly shows, and what Cognaptus infers

A clean business reading requires separating evidence from interpretation.

That distinction prevents a common benchmark abuse: turning a research result into a purchasing slogan. ConDiFi is not saying “use Model X for finance.” It is saying “stop pretending finance reasoning is one thing.” That is less convenient, therefore more likely to be true.

Where the benchmark is strong

ConDiFi has three strengths.

First, it uses finance-native tasks. The divergent scenarios ask for branching financial futures, not toy creativity exercises like “list uses for a brick.” The convergent questions include regulatory, quantitative, cross-sectional, and temporal traps. This matters because financial reasoning is domain-shaped. A model can be excellent at general explanation and mediocre at weighting sector exposure under policy uncertainty.

Second, the benchmark is designed to reduce memorisation. The scenarios are based on post-1 May 2025 developments. That does not eliminate leakage, but it is more thoughtful than recycling widely available benchmark items and then acting surprised when models have seen them.

Third, the paper tries to measure structure. The Richness metric is not perfect, but the instinct is right. Creativity in finance is not just novelty of phrasing. It is the ability to produce a structured map of possible futures. A narrow but eloquent answer is still narrow.

Where the benchmark should not be overread

The limitations matter because they affect how the benchmark should be used.

The divergent track relies on GPT-4o as judge for four of the five dimensions. That introduces judge-model bias, even with penalty rules. A model can be scored partly according to another model’s preferences about what “good” financial reasoning sounds like. Human audits are not decorative here; they are necessary.

The Richness metric captures tree breadth and depth, but it may reward wide shallow branching. The paper reports an approximate correlation of 0.56 with human ratings, which is useful but not definitive. A model can generate many branches and still fail to identify the one that matters.

The scope is also bounded. The dataset focuses on U.S. and major-market equities. Actionability is interpreted through U.S.-centric financial norms. The results may not transfer cleanly to private credit, emerging markets, commodities, crypto, real estate, insurance, or local regulatory environments.

The model set omits important closed models, including Claude and Gemini, and does not test domain-fine-tuned financial systems. The prompting setup is deliberately minimal: single-shot prompts, no tools, no few-shot examples, fixed decoding, and a 4096-token output ceiling. In production, a well-designed finance agent would likely use retrieval, calculators, market data tools, memo templates, policy constraints, and multi-step critique loops. ConDiFi tests base reasoning under controlled conditions, not full workflow performance.

There is also a small reporting ambiguity worth noting. The paper states 607 divergent scenarios and 14 evaluated models, while also reporting 9,380 divergent samples for analysis. Those numbers do not obviously reconcile by simple multiplication. It does not overturn the main findings, but it is a reminder that benchmark papers, like earnings calls, deserve arithmetic attention.

How finance teams should use this

The operational takeaway is to build evaluation gates around reasoning mode.

For divergent work, do not ask only whether the model’s answer is plausible. Ask whether it produces multiple materially different futures. Score the answer for novelty, causal depth, second-order implications, and decision triggers. Penalise generic macro wallpaper. Nobody needs a model to say “uncertainty may increase.” That sentence should be taxed.

For convergent work, create adversarial cases where the attractive answer is wrong. Include conflicting signals, delayed effects, regulatory sequence constraints, and negative information that competes with positive milestones. Measure whether the model selects the entailed answer, not the most comfortable one.

For governance, log the failure mode. “Wrong” is too vague. Was it a nuance miss, a factor-weighting error, an optimistic bias, a causal sequencing failure, or a task misunderstanding? Different failures need different controls. A nuance miss may require better extraction. A weighting error may require explicit factor scoring. Optimism bias may require downside-first prompting and human review.

For procurement, compare models by workflow role:

This is less tidy than buying one model and declaring victory. But tidy thinking is not the same as risk management. Finance allegedly knows this.

The real message: fluency is not fiduciary reasoning

The misconception ConDiFi attacks is common: if a model sounds like a financial analyst, it must be useful as one.

The paper’s results say otherwise. GPT-4o can score well on plausibility while lagging the top divergent models on novelty and actionability. Llama4 Maverick can lead convergent correctness while ranking much lower on open-ended scenario generation. Cohere Command A can excel in divergent foresight while sitting mid-pack on refined convergent selection. DeepSeek-R1 looks unusually strong across both, but still does not dominate every track.

This is what mature AI evaluation should look like. Not a leaderboard beauty contest. A task map.

The next phase of financial AI will not be won by models that merely write convincing paragraphs. It will be won by systems that know when to diverge, when to converge, when to challenge optimism, and when to admit that a branch is decorative rather than decision-relevant.

Finance has always needed two minds: one to imagine what could happen, and one to discipline that imagination before money moves. ConDiFi gives us a way to test whether LLMs have both. Most do not. Some are getting closer. None should be left unsupervised with the cheque book.

Cognaptus: Automate the Present, Incubate the Future.