Remembering Why Is Not Proving Why

TL;DR for operators

A research agent can preserve an experiment without preserving why its result might hold. CauProphet¹ addresses this by storing reusable causal hypotheses with supporting evidence, contradictions, and applicability conditions, then adapting them to new tasks.

Across two chronological financial factor studies, the system improves mean next-period RankIC by 25.4% relative to ReAct with text memory, although it performs worse in the volume-only study. It also improves pre-resolution forecasting accuracy and intervention-effect estimation in selected benchmarks. These are task-performance results, not proof that its stored causal relationships are correct.

The additional architecture has a measurable cost: nearly twice the token consumption of text-memory ReAct in the financial studies. For teams building persistent research or decision-support agents, the central question is whether maintaining revisable explanations improves downstream decisions enough to justify the additional computation, complexity, and validation requirements.

A saved factor does not preserve its explanation

Consider an investment research agent that discovers a promising stock-selection factor. It saves the formula, historical performance, and research notes. Months later, another research task retrieves those records.

The agent can recover what worked. Whether it can establish why the factor was expected to work, which market conditions supported that explanation, and what new evidence would invalidate it is a different question.

This distinction determines what an agent can carry forward between investigations. A searchable transcript preserves observations and decisions. A reusable explanatory record can additionally preserve proposed mechanisms, their assumptions, and the evidence that supports or challenges them.

Kaiwen Wu and Chaochao Lu’s CauProphet explores this second approach. Its design treats previous causal explanations as persistent but revisable knowledge, rather than conclusions that should automatically govern future tasks.

Persistent explanations and working beliefs serve different purposes

CauProphet divides causal knowledge into two layers.

Causal Cognition is the persistent layer. It contains a graph of recurring factors and proposed directed causal relationships, supplemented by versioned natural-language explanations. Each relationship can carry mechanisms, provenance, supporting and contradictory evidence, contextual conditions, time scales, and confidence information.

Causal Belief is the task-specific layer. When a new problem arrives, the agent retrieves potentially applicable relationships, maps abstract factors onto current variables, and combines previous explanations with present observations and assumptions.

The separation matters because a relationship relevant to one market, period, or intervention may be inappropriate elsewhere. The agent must decide which retained explanation applies before using it to investigate or recommend an action.

Knowledge revision is part of this process. New observations can change a working belief, and selected findings can subsequently be incorporated into persistent Causal Cognition. The system therefore supports continuity without requiring every earlier conclusion to remain unchanged.

CauProphet implements this process through several complementary mechanisms.

First, it aggregates evidence for individual causal relationships. Supporting claims increase a relationship’s log-odds score; contradictory claims reduce it. Low-, medium-, and high-confidence evidence receives fixed positive weights of 0.5, 1, and 2, respectively. These updates are applied to claims associated with the same directed relationship, context, and time scale, with safeguards against repeatedly counting the same source episode.

The calculation resembles Bayesian evidence updating but approximates likelihood ratios using predefined confidence weights. Its resulting probability-like scores help prioritize hypotheses for retrieval; they are not demonstrated to be calibrated probabilities of causal truth.

Second, the system addresses competing explanations. Several causal structures may fit the same observations, so Causal Scout uses answer set programming to enumerate candidate graphs subject to required or forbidden edges, temporal constraints, and acyclicity. Candidates can then be assessed using confidence, simplicity, statistical evidence, and observations that might distinguish them.

Third, external tools support causal discovery, adjustment-set analysis, effect estimation, and reliability checks. A partially observable decision-process perspective organizes how the agent investigates and acts under incomplete information.

Together, these mechanisms create a workflow in which retained hypotheses inform current investigation, new evidence challenges those hypotheses, and selected conclusions become available to subsequent tasks.

The architecture makes causal assumptions inspectable and revisable. It does not independently establish that those assumptions identify genuine causal effects.

Sequential research improves, but not uniformly

The clearest test of persistent knowledge is a sequence in which earlier findings can influence later decisions.

CauProphet’s chronological AlphaBench-derived financial experiments use Chinese CSI300 price and volume data. Agents develop factor formulas during successive half-year periods and submit them before the following evaluation outcomes become available.

The main comparison measures next-period RankIC: the cross-sectional correlation between factor rankings and realized next-day stock-return rankings, reported after multiplication by 100.

Source: Section 4.2, Table 1; Appendix C.1.1, Tables 5–6. Higher RankIC is better.

The combined advantage is approximately 25.4% relative to the text-memory baseline. This is an improvement in predictive ranking correlation, not a 25.4% increase in investment returns.

The split across studies is revealing. CauProphet performs substantially better in the price study but falls behind text memory in the volume study. Better explanatory memory therefore does not guarantee better factor selection under every research condition.

An additional five-round factor-search comparison provides evidence about repeated improvement. With Causal Cognition, held-out RankIC×100 rises from 4.213 in round one to 4.850 in round four, before finishing at 4.820. The text-memory condition remains at 4.191 throughout.

That comparison is consistent with useful accumulation of research knowledge, but it follows one continuing sequence per memory condition. The text-memory agent also attempts progressively fewer new candidates. It does not establish a general learning curve across independent runs.

Forecasting and interventions test different capabilities

The financial experiments test whether knowledge can improve sequential research. Other evaluations ask whether the architecture helps agents update predictions or choose actions under causal uncertainty.

In FutureSim, 24 forecasting questions are replayed with historical news access. Each system produces 171 pre-resolution predictions among 195 scored question-day predictions.

CauProphet achieves 28.6% pre-resolution Top-1 accuracy, compared with 21.3% for ReAct and 22.0% for OpenCode. Its pre-resolution Brier Skill Score is 0.080, versus 0.049 and 0.035, respectively.

These main forecasting results favor CauProphet during the period when events are still developing. At the final snapshot, however, all three systems reach 29.2% Top-1 accuracy, and ReAct has the higher Brier Skill Score: 0.092 versus CauProphet’s 0.085.

The advantage is therefore more evident in evolving forecasts than in final-snapshot prediction quality.

The paper also evaluates three synthetic structural causal models with known generating mechanisms. Across the five-node nonlinear, eleven-node nonlinear, and ten-node linear models, CauProphet reports the lowest mean intervention-effect relative errors among the evaluated methods: 1.1%, 7.4%, and 8.1%. It identifies the correct effect direction in all nine runs.

These known-ground-truth tests provide stronger evidence about causal estimation within the specified synthetic models, but each model is evaluated only three times. Their results cannot establish reliability across unrestricted observational problems.

In CausalGame, which tests intervention decisions in simulated drone environments, CauProphet wins all three runs in each of two trap scenarios. It achieves no victories in the weather-noise scenario, where the other configurations also fail to win.

Two aggregate ablations favor the complete system: one jointly removes Causal Cognition, Causal Belief, and Causal Scout while retaining causal computation tools; the other removes dedicated causal capabilities. These tests support a system-level contribution, but cannot isolate the independent effect of causal memory or any other individual component.

Broader application tests extend the picture. On 12 BLADE scientific-analysis tasks, CauProphet ties ReAct on statistical-model reference matching and improves value-transformation matching from 40.3% to 48.5%. In 16 selected AppWorld tasks, it passes 173 of 174 application checks across 23 executions. The latter is a demonstration of task execution rather than comparative superiority, because matched baseline scores are not supplied for that subset.

The financial advantage carries an unequal compute bill

The chronological financial experiments expose a limitation that directly affects the architecture decision.

In price research, CauProphet consumes 211,247 tokens, compared with 116,299 for ReAct with text memory. In volume research, the corresponding figures are 233,169 versus 120,496.

CauProphet also tests 42 candidates in each study, compared with 36 for the text-memory baseline. Model-call totals differ as well.

Consequently, the financial comparisons cannot separate the contribution of structured causal knowledge from that of additional computational and candidate-search effort. The higher aggregate RankIC is a valid observed result under the reported configurations, but it is not an estimate of the isolated return from causal memory at equal cost.

There is a second unresolved issue: knowledge quality over time.

A persistent causal hypothesis can become obsolete, accumulate misleading evidence, or be transferred into an inappropriate context. The paper’s approximate evidence aggregation organizes claims and contradictions, but its fixed weights and conditional-independence assumptions do not establish empirical confidence calibration. When observations are insufficient, the system also depends more heavily on the underlying model’s prior knowledge.

The reported experiments cover relatively short sequences. They do not yet establish long-term stability, production reliability, or cost effectiveness in changing environments.

The architecture decision belongs in downstream evaluation

For an investment research team, the immediate application is to store each factor hypothesis alongside its mechanism, historical tests, market conditions, and contradictory observations. When new data arrive, the agent could evaluate whether the earlier explanation remains applicable before selecting another research direction.

For a forecasting team, retained hypotheses could be updated as news develops, with performance assessed across the prediction horizon rather than only at event resolution.

For an analytics team, explicit causal assumptions could inform variable selection, adjustment decisions, and requests for further evidence. These are Cognaptus application inferences, not independently demonstrated production outcomes.

A fair deployment-oriented evaluation should compare the full causal architecture with simpler searchable transcripts, text memory, and structured hypothesis records under matched token and candidate-search budgets. It should also examine whether individual causal components improve decisions, whether contradictory evidence changes behavior appropriately, and whether stale hypotheses are transferred across incompatible conditions.

The relevant measures extend beyond benchmark scores: decision quality, unsupported causal claims, revision accuracy, computational cost, and performance under changing conditions all affect whether the additional infrastructure is justified.

Remembering explanations is a design choice, not causal verification

CauProphet demonstrates that an agent can maintain causal hypotheses across tasks, apply them to current circumstances, compare alternative explanations, and revise what it retains. Its benchmark results provide moderate support for the value of this complete approach in several evaluated settings.

The unresolved question is how much of that value comes specifically from persistent causal knowledge, rather than additional tools, search effort, or other parts of the architecture.

For organizations developing long-lived agents, the paper makes a credible case for testing explanatory memory. The next requirement is to establish when those explanations improve decisions at comparable cost—and when they should no longer be trusted.

Cognaptus: Automate the Present, Incubate the Future.