When Old Instructions Outlive Their Authority

TL;DR for operators

An AI agent can retain every instruction in a conversation and still execute the wrong task. The problem becomes acute when users replace requirements, withdraw constraints, or revise decisions on which earlier work depended.

A controlled study introduces IntentFlux, a benchmark that measures this failure under changing user instructions.¹ In a turn-matched experiment, mean task score falls from 0.734 without revisions to 0.469 with revisions. Additional conversation length alone cannot explain the difference.

The proposed StateForge system partially repairs this problem by maintaining active requirements and invalidating conclusions derived from obsolete ones. Its full implementation improves mean benchmark score from 0.367 to 0.467. Yet supplying the correct final requirements only raises performance to 0.549, below the clean single-turn score of 0.778.

For enterprise operators, the implication is architectural: current requirements need to be distinguished from conversation history, and downstream work must respond to changes. However, matched experiments do not establish that StateForge’s particular state representation is better than equally resourced recap methods. The investment decision requires measuring execution accuracy and computational cost together.

The conversation was equally long. The task was not equally successful.

Consider a procurement assistant preparing a supplier comparison. The user initially specifies one delivery deadline, later changes it, and eventually withdraws a supplier restriction. The assistant may preserve the entire conversation yet continue evaluating suppliers against a deadline or restriction that no longer applies.

The failure concerns which instructions remain authoritative. An intermediate shortlist could also survive after the requirement that produced it has been withdrawn.

The paper calls this intent drift: an agent’s failure to discard superseded or withdrawn user requirements when completing the user’s current task.

The central experimental comparison is revealing.

Source: Section 5.1 and Appendix C, Table 10. The tested model is Qwen3.5-122B on 125 General-Test tasks; scores are averages of normalized task-native grades.

The no-revision control presents the complete final requirements initially, followed by neutral confirmations. It matches the number of dialogue turns in the evolving-intent condition and uses comparable message lengths.

The 0.265-point advantage over the revised-instruction condition therefore challenges a simple explanation based on turn count. However, the experiment does not independently isolate the effects of revision wording, message ordering, withdrawn constraints, or other differences in the dialogue.

For operators evaluating long-running agents, this distinction matters. A model’s ability to handle lengthy conversations is not sufficient evidence that it can correctly handle changes to the task itself.

IntentFlux makes obsolete instructions consequential

Measuring this problem requires knowing what the agent should do after every revision.

IntentFlux begins with executable source tasks drawn from seven families, including code generation, mathematical reasoning, SQL, data-to-text generation, summarization, and function calling. It transforms these tasks into evolving conversations while preserving their final intended outcomes and original grading procedures.

Requirements are represented as atomic goals modified through three operations:

• Add: introduce a requirement.
• Delete: withdraw a requirement.
• Replace: substitute a new requirement for an existing one.

The benchmark also constructs variants, which are plausible values subsequently replaced, and decoys, which are plausible constraints subsequently withdrawn. Decoys are retained only when following them changes the original task grader’s verdict.

That design makes outdated instructions potentially consequential to measured task success, although a failed task does not necessarily prove that the agent followed a particular obsolete instruction.

Across eight evaluated models on the fixed 125-task General-Test set, clean-condition full-credit completion exceeds hard-drift completion. The clean-minus-drift gap ranges from 0.344 to 0.448, with positive paired 95% confidence intervals for every model.

A separate calibration repeatedly evaluates 627 source tasks at increasing construction difficulty:

Source: Appendix B, Table 5.

The mean-score decline from easy to hard is approximately 19.3% relative to the easy condition. But variants, decoys, and dialogue length increase together; the experiment cannot attribute that reduction separately to each factor.

Task families also behave differently. LiveCodeBench loses 0.309 mean-score points across the calibration, whereas BFCL gains 0.048. The broad finding is degradation under the benchmark’s evolving-intent conditions, not a universal monotonic response in every task type.

Remembering the conversation does not determine what remains valid

Conventional context management addresses information availability. It compresses or summarizes dialogue so the agent can retain relevant history.

That process does not inherently establish whether a retained requirement remains active.

In the paper’s principal General-Test harness comparison, rolling summarization scores 0.354 and context compaction scores 0.392, compared with 0.367 for bare multi-turn execution. Neither tested method restores performance near the clean-condition score.

StateForge instead implements state folding: it maintains an explicit estimate of current requirements, applies additions, deletions, and replacements, and invalidates derived conclusions when their supporting requirements change.

For the procurement assistant, this would mean updating the active deadline and reconsidering any shortlist dependent on the withdrawn supplier restriction. That example illustrates the proposed mechanism; the paper does not test a production procurement deployment.

The full StateForge harness raises mean General-Test score from 0.367 to 0.467, an improvement of 0.100 points with a paired 95% confidence interval of [0.030, 0.171]. Full-credit completion rises from 0.264 to 0.368.

This supports the evaluated system as a partial mitigation. Attribution to state folding alone requires more care: StateForge also includes a format tracker and a final output gate.

A separate component-matched re-run is particularly informative.

Source: Appendix D, Table 12. This is a separate protocol from the headline 0.467 result.

The re-run matches model-call and token budgets. Neither recap baseline differs significantly from full StateForge under the reported paired confidence intervals.

Consequently, the evidence establishes a benefit for the complete system in its principal comparison, but does not establish that its explicit state representation uniquely outperforms computationally matched recap methods.

Even perfect knowledge of current requirements leaves errors

An agent can fail because it reconstructs the wrong task. It can also correctly identify the task and then produce an incorrect answer or action.

The authors examine this distinction through oracle final-state injection: the agent receives the ground-truth final requirements while the earlier dialogue remains available.

In the principal General-Test comparison:

• Bare multi-turn execution scores 0.367.
• StateForge scores 0.467.
• Oracle final-state injection scores 0.549.
• Clean single-turn execution scores 0.778.

Better state estimation therefore offers additional headroom, but exact final-state information does not recover clean-condition performance.

The remaining 0.229-point gap is a particularly consequential boundary for system designers. Correct requirement tracking cannot be treated as a complete substitute for reliable downstream reasoning and execution.

The interactive VitaBench evaluation adds another qualification. Without a closing restatement of final intent, the agent’s partial-credit rubric score falls from 0.697 in the clean condition to 0.594. Restating final intent at the end raises the drift-condition score to 0.720, potentially masking earlier tracking difficulties. Full-credit completion remains 0.190 across these three non-oracle conditions.

Evaluation protocols must therefore distinguish independently reconstructing current intent from receiving it again immediately before execution.

Smaller trackers offer another design option

State maintenance need not necessarily use the same large model that performs the downstream task.

In the unified tracker-scale re-run, a 9B-parameter tracker scores 0.399 against 0.428 for the 122B reference. The paired confidence interval includes zero, although this is not proof of equivalent performance.

The paper also explores training-based alternatives.

One iteration of on-policy distillation improves a 2B tracker configuration’s reported held-out full-episode score from 0.266 to 0.445 while keeping the 122B downstream generator frozen. The appendix identifies this as a tracker-training evaluation protocol distinct from the main General-Test task-score comparison.

Separately, thinking distillation improves a standalone 35B agent’s drift-condition score from 0.296 to 0.461, while its clean-condition score declines from 0.745 to 0.718.

These experiments suggest two possible deployment strategies: adapting an independent requirement tracker or internalizing the behavior within the task-solving model. Their scores are not directly comparable because the configurations and evaluation protocols differ.

Enterprise deployment requires more than a state tracker

Cognaptus interpretation: For enterprise workflows, the architectural opportunity is to separate three responsibilities: establishing current requirements, identifying dependent work invalidated by revisions, and verifying subsequent execution.

A procurement team could evaluate whether revised constraints propagate into supplier screening and approval documents. A customer-service operator could test whether withdrawn instructions remain excluded before an external action. A scheduling assistant could check whether a revised time invalidates earlier booking decisions.

Before adding a dedicated tracking model, the organization should compare it against a resource-matched recap baseline using revised-instruction accuracy, dependent-work invalidation, downstream task success, latency, and cost.

The research leaves meaningful boundaries. Its benchmark uses constructed dialogues and selected task-native graders; a sampled simulator audit reports 96% fidelity and 97% coherence but cannot validate every interaction. The principal harness and interactive experiments use one tested backbone, and the benchmark does not establish reliability in unrestricted production workflows.

These limitations affect deployment claims, not the core benchmark observation.

The evidence identifies a measurable failure associated with changing user requirements and demonstrates partial recovery through a revision-aware agent system. It also shows that correct state information and successful task execution remain distinct problems.

An agent that remembers an old instruction must still determine whether that instruction has authority. An enterprise system must then ensure that its decisions and actions reflect the resulting current specification.

Reference

Cognaptus: Automate the Present, Incubate the Future.