The Post-Call Forecast That Starts Before the Call Ends

TL;DR for operators

An earnings-call model can pass a chronological train-test split while being evaluated against returns that began before its complete input could have been available. In a 552-call audit, Ma and colleagues¹ find that moving the return start from documentary call-start time to reconstructed audio completion plus a stipulated 15 minutes reverses 9.4% of return signs and changes 28.3% of fixed-cutpoint quintile labels.

A separate evaluation of 1,710 Dense Earnings Call events finds that correcting selected before-market target windows raises three-day mean squared error (MSE) by 18.1–21.4% across four frozen prediction streams, even though their point rankings remain unchanged.

For financial AI teams, the practical requirement is an Availability Contract: specify what information the model uses, when that information could have been ready, which prices define the outcome, and what forecasting claim those records justify. The study establishes substantial benchmark sensitivity, not verified historical forecast readiness or profitable trading performance.

A forecast can be evaluated against prices that moved before its input was complete

A quantitative investment team receives an earnings-call NLP benchmark with promising forecasting results. The data are chronologically separated, the targets are described as post-call returns, and the model comparison produces a stable leaderboard.

Before approving the result, the team needs to establish when the measured price movement actually began. An earnings call releases information progressively. A representation containing the complete Q&A includes statements that were not available when the call started.

If the return window begins at that earlier point, the evaluation includes price changes occurring before the model could have received the complete representation. Chronological separation between training and test calls does not resolve this within-call mismatch.

The distinction affects what the benchmark measures. A broad event-window return can be appropriate for studying how market prices relate to earnings disclosures. Evaluating a forecast made after the complete Q&A requires a different starting boundary.

The Availability Contract connects model inputs to outcome timing

The paper introduces a nine-field Availability Contract specifying the evidence needed to interpret an evaluation result.

It records the permitted claim, input boundary, rule for constructing an input-completion proxy, processing allowance, model checkpoint, inference-completion evidence, price-anchor rule, outcome endpoint, and sample-selection rule.

The authors demonstrate the approach using three clocks applied to the same earnings calls:

For S and A, the selected price is the closing price of the first regular-session five-minute bar whose start is strictly later than the relevant timestamp. The mapping uses the New York Stock Exchange calendar, including early closes.

To construct A, the authors add the longer of decoded MP3 duration and the final finite sentence-alignment endpoint to the documentary source-start timestamp. They then add a stipulated 15-minute processing allowance.

This produces a reproducible completion proxy, not a verified record of transcript delivery or model inference.

The main comparison keeps each call, its retrospective NLP representation, and the following-session return endpoint fixed. Only the starting price changes. This isolates the consequences of alternative return definitions without changing the underlying observations.

High return correlation still permits substantial label reassignment

The primary audit contains 552 complete earnings calls from 24 large U.S. firms during 2018–2024.

Moving from S to A produces a Spearman return correlation of 0.900. That high aggregate agreement nevertheless conceals meaningful changes in individual labels.

Source: Section 5.1, Table 3 Panel A. The comparisons use the same 552 calls and common return endpoint.

The two quintile measures answer different questions. Relative quintiles are recalculated separately for each clock’s return distribution. Fixed-A quintiles apply the same A-clock cutpoints throughout, showing how many observations cross stable category boundaries.

The S-to-A price anchors differ for 311 of the 552 calls. Within that subset, 16.7% of return signs and 50.2% of fixed-A quintile assignments change. All 206 after-close calls have identical S and A anchors and numeric returns, although their relative-quintile assignments can differ when the reference distribution changes.

The larger L-to-A change also shows why event-date alignment and complete-input alignment should be audited separately. Most of the total previous-close adjustment is not attributable solely to the interval between documentary call start and reconstructed completion.

For a research lead evaluating a directional classifier or return-ranking strategy, these results imply that the benchmark’s outcome definition can alter which observations count as successful forecasts. They do not establish which alternative clock would produce superior trading returns.

A stable leaderboard can conceal higher forecasting errors

The paper tests a different consequence using the public Dense Earnings Call (DEC) benchmark.

This independent experiment contains 1,710 earnings events and four frozen volatility-prediction streams. The authors move the target windows for 1,191 before-market events forward by one trading session, leaving after-market targets and all predictions unchanged.

The exercise is a frozen-prediction target sensitivity test, not a retraining experiment.

At the three-day horizon, the revised targets produce:

• 58.8% relative-quintile reassignment and 43.5% fixed-cutpoint reassignment.
• 18.1–21.4% higher MSE across the four prediction streams.
• Identical point leaderboard rankings under MSE, mean absolute error, and Spearman correlation.

The rankings also remain unchanged at seven- and fifteen-day horizons, where MSE increases are smaller: 8.1–10.1% and 3.8–6.2%, respectively.

Ranking stability is therefore not equivalent to loss stability. Every predictor can become less accurate against revised targets while preserving its relative position.

Nor does an unchanged point leaderboard imply certainty about the best model. In three-day issuer-level and calendar-quarter resampling, the originally best-MSE stream remains best in approximately 67% and 72% of draws, respectively.

For a model-risk committee comparing vendors, the decision consequence is specific: retain the published leaderboard as a relative comparison, but require absolute losses and ranking stability to be reassessed under a justified target clock before treating its reported performance as validated.

The removed return interval explains changes in text-return associations

The authors also examine four retrospective Q&A representations: dictionary-based lexical tone, FinBERT sentiment, Qwen3-4B answer valence, and E5-based narrative revision.

These regressions are exploratory rather than the paper’s principal predictive evaluation.

Because log returns are additive, changing a return’s start while holding its endpoint fixed removes exactly the intervening price movement. With identical observations, predictors, controls, and fixed effects, ordinary least squares preserves this decomposition in its coefficients.

The Qwen answer-valence example illustrates the result:

The overall L-to-A coefficient difference is +1.355 percentage points per standard deviation. Of this, +1.256 comes from the L-to-S interval and +0.098 from S-to-A.

Thus most of this particular change is associated with price movements preceding the documentary call-start anchor, rather than movements between documented start and reconstructed completion.

The decomposition is exact under the common-design OLS conditions. It is not a causal identification result. The text representations were constructed retrospectively, and the authors classify their representation-level inference as exploratory because construct inspection preceded the final testing specification.

Its methodological value is diagnostic: a quantitative researcher can identify which removed price interval accounts for an apparent text-return association instead of treating the coefficient change as unexplained model instability.

Turning the contract into an evaluation requirement

For financial AI teams, Cognaptus draws three operational implications from these results.

First, specify the prediction boundary before constructing labels. The benchmark owner should identify whether the intended task concerns an event response, a partial disclosure, or a representation containing the entire call. Representations with different information boundaries should either use a common later justified outcome anchor or be evaluated as explicitly different prediction tasks.

Second, retain a reproducible timing record. Dataset engineers should version source timestamps, audio-duration rules, processing assumptions, model checkpoints, market-calendar mappings, price-bar selection, endpoints, and sample attrition. A benchmark release should expose the full nine-field contract rather than only a target formula.

Third, audit labels before retraining. Quantitative validation teams can recalculate signs, categorical targets, and regression associations across candidate clocks. When fixed model predictions are available, rescoring them against revised targets provides a relatively direct measurement of benchmark sensitivity without conflating the result with model adaptation.

The paper’s claim ladder gives the approval committee a practical distinction:

The reported audit primarily reaches the second level. Moving to a later return clock is not, by itself, evidence that the higher levels have been satisfied.

What the reconstructed clocks cannot establish

Several limitations directly constrain how the findings can be used.

The documentary start timestamps largely come from archived page headers rather than independently verified call starts. Audio completion is reconstructed, and the 15-minute processing allowance is assumed. Varying that allowance from zero to 120 minutes changes S-to-A sign-reversal rates from 6.5% to 15.0%, demonstrating that the selected readiness assumption materially affects the result.

Sample availability also matters. Only 552 of the original 2,094 calls survive the full set of timing, audio, price, and representation requirements. The remaining sample is disproportionately concentrated in later years and larger firms.

The retrospective Qwen analysis used a 2025-vintage checkpoint in June 2026 without an established historical knowledge cutoff. Actual transcript production and inference completion were unobserved. Selected five-minute closing prices are label anchors, not verified executable prices.

As an endpoint sensitivity test, the authors also examine returns ending after exactly 390 exchange trading minutes. The exploratory Qwen A-clock coefficient is −0.226 percentage points per standard deviation, compared with −0.233 under the common following-session close. This comparison addresses holding-period sensitivity but does not reproduce the exact common-endpoint decomposition, and the secondary fixed-duration coefficients remain nonsignificant after multiplicity adjustment.

These boundaries leave the primary measurement findings intact while preventing a stronger conclusion about historical deployability, causal predictability, or investment profitability.

Validate the outcome clock before approving the forecast

The study changes how an earnings-call benchmark should be assessed. A chronological train-test split addresses separation between observations; it does not demonstrate that every evaluated return begins after the information used to produce its prediction became available.

The Availability Contract turns that missing requirement into an auditable specification. Its value is not that every benchmark must use the latest possible return start, but that each benchmark must document the timing assumptions appropriate to its claimed task.

For financial AI evaluation, a stable model ranking is insufficient when the target itself changes under a defensible alternative information-availability boundary. The first approval decision should concern what the benchmark actually measures. Only then can its results be interpreted as evidence about forecasting performance.

Cognaptus: Automate the Present, Incubate the Future.