When the Evidence Changes, Does the Agent?

TL;DR for operators

An AI agent can finish a workflow correctly, explain its decisions, and recover after retries without demonstrating that it can respond appropriately when information changes.

Sejin Kim and Sundong Kim propose evaluating adaptive reasoning through two complementary capabilities: transferring applicable rules across related tasks and revising decisions when new evidence invalidates an earlier assumption.¹

Their framework calls for controlled changes to task structure or evidence, with comparisons against cases where information remains unchanged or should not affect the decision. Evaluators must also account for test-time computation so that more search is not mistaken for better adaptation.

For enterprise AI teams, this suggests extending acceptance tests beyond completion rates. The relevant evidence is whether agents change the decisions that new information affects while preserving those still supported.

The boundary: This is a conceptual evaluation framework, not an experimentally validated benchmark. Its ability to predict production reliability or justify additional testing costs has not been established.

A successful workflow can conceal an inability to revise

Consider an enterprise agent processing a customer request. It retrieves the relevant policy, checks the customer’s account, and selects the correct action. The workflow passes its acceptance test.

During a subsequent execution, a tool returns new information that invalidates one assumption behind that action. The agent might revise the affected decision, preserve everything else, and continue. It might instead repeat its search, generate another plausible answer, or unnecessarily change decisions unrelated to the new evidence.

A conventional completion score may not distinguish these behaviors. Even if the eventual answer is correct, it does not establish that the agent responded to the contradiction.

Kim and Kim identify this limitation across five familiar sources of reasoning evidence. Outcome evaluation measures success; reasoning traces expose expressed reasoning; process evaluation assesses intermediate-step quality; resource measurements describe computational effort; and human-relative evaluation provides efficiency comparisons. Each supplies information, but none independently establishes evidence-sensitive adaptation (Table 1, What Existing Reasoning Evaluations Tell Us).

This matters when AI procurement teams qualify agents for workflows whose inputs, constraints, and tool responses change during execution. An initial success rate describes performance under tested conditions. It does not tell operators which earlier decisions an agent will preserve or revise when those conditions change.

Adaptation has two distinct requirements

The paper separates adaptive reasoning into two behavioral properties that require different tests.

Across-task adaptivity concerns whether an agent carries an applicable abstraction from one task to another. Two tasks may look different while relying on the same underlying rule. Alternatively, a new task may change the underlying rule and require a corresponding change in strategy.

A system demonstrates stronger evidence of transfer when its behavior follows this distinction: preserving or predictably transforming an approach when the relevant structure remains intact, and modifying its approach when that structure changes.

Within-task adaptivity concerns revision during an ongoing task. An agent has an observable hypothesis, plan, program, or proposed action. New evidence arrives. The evaluation examines whether the implicated decision changes appropriately and whether subsequent behavior reflects that revision.

These capabilities are related but not interchangeable. An agent might successfully apply a rule across different cases yet persist with an invalid assumption inside a single execution. Another might respond to corrective feedback without reliably recognizing the same rule across differently presented tasks.

The authors formalize observable behavior as:

Here, behavior depends on the task $\tau$, evidence available by step $t$, and an explicit test-time resource budget $q$. The notation directs attention to the conditions under which behavior occurs, rather than treating a final output as a complete account of reasoning.

The framework does not claim that observed behavior reveals the internal computation responsible for it.

ARC-AGI illustrates what should change and what should remain stable

The technical supplement uses ARC-AGI examples to make the two distinctions concrete.

For across-task adaptation, the authors identify two existing ARC-AGI-1 tasks, 1e0a9b12 and 3618c87e, that share an evaluator-specified pixel-level downward-relocation abstraction. A third task, 25ff71a9, changes the manipulation unit to a connected object.

The proposed diagnostic asks whether behavior follows the relevant abstraction rather than superficial similarities between tasks.

These are illustrative relationships between existing tasks, not synthetically constructed matched interventions. The paper reports no measured model transfer performance on these examples.

The within-task illustration uses ARC-AGI-3 task r11L, an interactive environment involving movable endpoints. The supplement proposes branching from a common interaction history and introducing different evidence at the same decision point.

Source: Technical Supplement, Table 4. These are proposed intervention branches and expected responses, not observed experimental outcomes.

The distinction is operationally significant. Contradictory evidence warrants a targeted revision; confirming or irrelevant evidence should preserve supported components.

An agent that changes its entire strategy after every interaction could appear responsive while actually exhibiting unstable behavior. Conversely, an agent that repeatedly generates answers without updating its implicated hypothesis may eventually succeed without supplying strong evidence of within-task adaptation.

A different answer is not sufficient evidence

The framework proposes three complementary criteria for interpreting behavioral changes.

Relevance: Does the response depend on information that matters to the decision?

Specificity: Does the change target the hypothesis, action, or plan component implicated by the new information?

Consistency: Do comparable interventions produce comparable response patterns?

Together, these criteria require evidence of appropriate stability as well as appropriate change.

A credible evaluation therefore needs controls. Across tasks, evaluators should compare variations that preserve the relevant concept with variations that change it, while specifying what else remains constant. Within a task, they should branch from the same preceding history rather than compare executions that have already accumulated different experiences.

Negative controls involving irrelevant evidence help identify indiscriminate changes. Confirming evidence tests whether supported hypotheses persist. Repeated runs help distinguish systematic responses from stochastic variation.

Test-time resources require equal attention. An agent allowed more samples, tokens, search steps, refinement rounds, or tool interactions may achieve better outcomes simply because it explores more possibilities.

Additional search is not necessarily undesirable. But without matched or transparently reported budgets, evaluators cannot confidently attribute improved outcomes to evidence-sensitive adaptation.

The authors position these procedures alongside existing approaches such as Contrast Sets, CheckList, ConceptARC, and Belief-R (Technical Supplement, Table 3). Controlled intervention testing is not itself a new invention. The proposed contribution is to organize abstraction transfer, within-task revision, and resource accounting within a common reasoning-evaluation framework.

Enterprise acceptance tests can examine selective revision

Cognaptus inference: Enterprise AI teams can translate this framework into more diagnostic acceptance tests for agents operating in changing environments.

Consider a customer-service agent that has confirmed a customer’s identity and prepared a recommendation based on account information. A later tool response changes the customer’s eligibility for one service.

An evaluator can preserve the initial workflow state, introduce the updated eligibility information, and inspect whether the agent modifies the affected recommendation while retaining the independently verified identity and other supported details.

The test becomes more informative when compared with branches containing confirming information, irrelevant account changes, and no additional evidence.

For an AI procurement team, these comparisons could supplement completion-rate benchmarks when deciding whether an agent is sufficiently reliable for a workflow involving corrections or changing constraints.

For a quality-assurance team, they could become regression tests after model, prompt, or tool changes. For incident investigators, recorded pre-intervention decisions, evidence updates, subsequent actions, and resource consumption could help distinguish persistent assumptions from unstable revision or repeated search.

The paper’s reporting recommendations (Table 2) support this approach: document the intervention, expected response, observed intermediate decision, subsequent outcome, and test-time budget.

The prospective benefit is better failure diagnosis, not an established reduction in production errors or operating costs. Whether these tests predict deployed behavior, and whether their diagnostic value exceeds their design and execution costs, requires empirical validation.

What intervention tests cannot certify

The framework has several important boundaries.

First, Kim and Kim do not run new controlled evaluations of model cohorts. There are no reported intervention success rates, confidence intervals, model rankings, or measured improvements attributable to the proposed methodology. The ARC-AGI examples demonstrate evaluation logic, not empirical effectiveness.

Second, intervention validity is difficult to guarantee. A supposedly concept-preserving variation might unintentionally change difficulty or introduce cues. A concept-changing variation might alter several relevant properties simultaneously. Either problem weakens attribution to the intended intervention.

Third, observable hypotheses and actions are imperfect proxies for internal reasoning. Even consistent, evidence-sensitive behavior cannot uniquely identify whether a system used an internal abstraction, a heuristic, or another computational procedure.

Finally, responsiveness to corrective evidence does not establish that an agent pursues desirable objectives, respects oversight, or satisfies safety requirements. Adaptive reasoning and alignment remain distinct evaluation targets.

These limitations do not negate the proposed framework. They establish what its measurements would mean and what additional validation would be needed before treating them as deployment qualifications.

Evaluate the update, not only the answer

A correct answer establishes that an agent succeeded under the conditions tested. To establish that its behavior adapts appropriately, evaluators need evidence about what happens when those conditions change.

The contribution of Kim and Kim is to make that additional evidence more precise: preserve relevant structure, intervene on relevant information, observe targeted decisions, introduce appropriate controls, and account for computational opportunity.

For enterprise AI teams, the immediate implication is a more discriminating testing question: when a fact changes, can the agent revise what that fact invalidates without disturbing what remains supported?

The paper supplies a methodology for investigating that behavior. Whether the methodology improves real-world selection and monitoring of AI agents remains an open empirical question.

Cognaptus: Automate the Present, Incubate the Future.