TL;DR for operators

A coding agent can remember an unfinished change without completing it. That gap matters when a repository task involves multiple dependent edits, tests, and evolving requirements.

StateHarness, proposed by Zou and colleagues,1 maintains structured project state outside the agent’s conversation history. It records current hypotheses, findings, repository relationships, procedure failures, and unresolved work. A compact working brief returns relevant information to the agent, while change-impact retrieval identifies potentially affected files.

With the same underlying Claude Sonnet 5 model, StateHarness improves RefactorBench cleanup-check accuracy from 87.0% to 95.4% and full-chain completion from 29.4% to 35.3%. It also improves performance on two other long-horizon coding benchmarks.

The qualification is operationally significant. ChainSWE pass-to-pass regressions rise from 16 to 23, despite better issue resolution. StateHarness also requires more tokens, tool calls, and execution time than ordinary Claude Code.

For engineering teams, the decision is whether structured project-state maintenance improves verified completion enough to justify its overhead. Persistent records should support, not replace, executable checks of unfinished work and regressions.

A successful edit can leave the repository inconsistent

Consider a coding agent modifying a function signature. It updates the implementation, runs a relevant test, and receives a passing result. Earlier in the task, however, it encountered another caller that still uses the old interface.

Whether the agent completes the refactoring now depends on more than its ability to write the correct code. It must retain the discovered relationship, recognize that the interface change affects another file, and return to that obligation before finishing.

Long execution histories make this difficult. Findings, rejected hypotheses, previous test outcomes, and deferred changes accumulate across many interactions. A conversation transcript may contain this information without representing which facts remain relevant or which obligations are outstanding.

The engineering question is therefore how to preserve the current condition of the project, not simply how to preserve more conversation history.

StateHarness approaches this as a runtime design problem. It adds external state maintenance around an existing coding agent without changing the model’s parameters.

Project understanding becomes an explicit runtime artifact

StateHarness maintains four agent-authored persistent records, each serving a different purpose.

Record Information preserved
Task state Current focus, observations, and working hypotheses
Event log Findings, decisions, edits, tests, and rejected assumptions
Relation notes Connections among modified files, callers, tests, documentation, and configuration
Procedure guidance Previous procedure failures and policies for using or avoiding procedures

A fifth artifact, the working brief, selectively returns the current focus, relevant guidance, recent events, and diagnostic alerts.

This separation matters because a complete interaction history and an actionable project state answer different questions. The history describes what happened. The maintained records describe what the agent currently understands, which relationships it has discovered, and what remains unfinished.

The working brief is synchronized through fixed rules rather than repeatedly asking another model to summarize the entire conversation. Older events remain in the full record even after they leave the compact view.

StateHarness also watches for behavioral signals such as repeated file reads, missing command-completion events, or prolonged neglect of the brief. When eligible signals accumulate, a separate diagnostic session can revisit older evidence and produce alerts.

These signals are imperfect proxies for execution difficulty. The runtime does not establish that repeated reads necessarily indicate a reasoning failure, and its record checks validate structure and history preservation rather than factual correctness.

The architecture nevertheless establishes a concrete mechanism for carrying project understanding forward without requiring the agent to reconstruct it from every previous interaction.

Change-impact retrieval connects an edit to unfinished consequences

StateHarness extends state maintenance beyond recalling previous findings.

For supported interface changes, definition-changing whole-file replacements, and source-file deletions, the runtime searches for potentially affected artifacts. It uses available syntax-tree information and textual matching to identify candidate callers, tests, documentation, and related files.

The working agent must inspect those candidates, determine which relationships matter, and record whether follow-up work has been completed or remains outstanding.

This converts a discovered dependency into a persistent obligation. A caller identified early in execution can remain visible even after subsequent edits shift the agent’s attention elsewhere.

The implementation has boundaries. Its documented retrieval coverage focuses on Python and Cython source files and supported plain-text documentation. Body-only edits ordinarily do not trigger the same retrieval path, and retrieved locations are candidates rather than verified dependencies.

Still, this mechanism offers a plausible explanation for the paper’s stronger cleanup results: the agent receives support for tracking consequences beyond the file it just modified.

Better cleanup and completion, with uneven reliability gains

The main empirical evidence comes from matched-model benchmark comparisons. StateHarness and ordinary Claude Code use Claude Sonnet 5 with matched task inputs, and outcomes are determined by benchmark checks or external verifiers rather than the agent’s self-report.

Outcome Claude Code StateHarness
RefactorBench full-chain completion 29.4% 35.3%
RefactorBench subtask completion 77.9% 81.4%
RefactorBench cleanup-check accuracy 87.0% 95.4%
RefactorBench import-check accuracy 82.7% 94.4%
RefactorBench documentation checks 90.9% 100.0%
RefactorBench call-site checks 93.3% 93.3%
LHTB mean reward 0.427 0.576
ChainSWE bug-fix rate 35.1% 39.5%

Source: Section 4.2, Table 1. Metrics have different units and should be interpreted within their respective benchmarks.

The RefactorBench results show why completion and cleanup deserve separate attention. Requested-refactoring check accuracy improves only from 94.6% to 95.9%, whereas cleanup accuracy rises by 8.4 percentage points. Import and documentation consistency account for notable gains, but call-site accuracy does not improve.

On Long-Horizon Terminal-Bench (LHTB), StateHarness also exceeds the LongHorizon-Harness comparison system’s mean reward of 0.525. This result covers five of six software-engineering tasks; the excluded task exceeded the execution budget. Individual comparisons are mixed, including a slightly weaker Commit0 result than LongHorizon-Harness.

ChainSWE evaluates 37 issue chains executed continuously within evolving repositories. StateHarness improves the issue-level bug-fix rate, while reported residual debt declines from 60 to 43 items.

Yet pass-to-pass regressions increase from 16 to 23 relative to Claude Code. Residual debt is also a supplementary measure with method-specific components, including stale explicit state records.

These results reject a simple inference that preserving more project information necessarily produces fewer regressions.

The paper’s execution examples clarify the distinction. In a sqlglot case, additional regression checks reveal a previously passing test broken by a broad parser change, leading to a narrower repair. In a Commit0 example, the agent records an unmet coverage requirement and subsequently raises branch coverage from 85.9% to 98.6%.

But an ALP paper-reproduction execution records an unmet performance target and still terminates without resolving it.

These are illustrative trajectories, not independent causal tests. Together they distinguish three requirements: remembering a problem, acting on it, and verifying that it is resolved.

The performance improvement has a measurable runtime price

Maintaining external project state adds work.

On RefactorBench, StateHarness increases average recorded token usage from 25.6K to 33.1K, tool calls from 81 to 110, and execution time from 383 to 529 seconds.

On ChainSWE, recorded tokens rise from 24.9K to 35.3K, while average execution time increases from 300 to 383 seconds.

The comparison changes when the alternative is LongHorizon-Harness, which uses manager, executor, and auditor roles. On ChainSWE, StateHarness records 35.3K tokens, 96 tool calls, and 383 seconds, compared with 208.0K tokens, 272 calls, and 2,214 seconds for LongHorizon-Harness. StateHarness also achieves a higher bug-fix rate: 39.5% versus 32.5%.

These are observed runtime-configuration comparisons, not compute-matched estimates of total inference cost. Auxiliary diagnostic and harness sessions are incompletely captured in the token accounting.

The study also reports no component-level ablation separating the contributions of persistent records, brief synchronization, change-impact retrieval, reminders, and diagnosis. Consequently, the complete framework’s performance improvement cannot be assigned confidently to any one mechanism.

Deployment should measure completed obligations, not stored memories

For engineering organizations running autonomous coding agents on sequential refactoring or maintenance tasks, Cognaptus draws three operational implications.

First, project-state records are worth evaluating when work accumulates interdependent decisions and unresolved changes. A structured record of findings, dependencies, and verification outcomes can provide more actionable continuity than conversation summaries alone.

Second, recorded obligations need executable completion criteria. When an interface changes, relevant callers and tests should be checked before accepting the final patch. When a requirement remains unmet, its recorded status should influence completion decisions. StateHarness provides reminders and completion guidance, but its documented execution shows that these do not guarantee follow-through.

Third, platform evaluations should jointly measure verified task completion, regression preservation, unresolved work, tokens, tool calls, and elapsed time. A higher bug-fix rate can coexist with more regressions and greater execution cost.

These recommendations are design inferences from benchmark behavior, not demonstrated production deployment outcomes.

The remaining uncertainties are consequential. All experiments use one working model. The maintained ChainSWE state persists within a continuous request, not across independently initiated coding sessions. The benchmark results do not establish general effectiveness across unrestricted repositories, languages, or development workflows.

Organizations seeking cross-session project memory or stronger autonomous completion guarantees would therefore need additional evaluation beyond the capabilities established here.

A project-state record improves continuity, not correctness by itself

StateHarness shows that an existing coding agent can perform better when its accumulated understanding is maintained as explicit, evolving project state. The largest practical signal comes from improved follow-up consistency in sequential refactoring, where a successful local edit may otherwise leave dependent artifacts unfinished.

Its results also establish a limit: knowing about an obligation is not the same as satisfying it.

For teams deciding how much runtime infrastructure to place around coding agents, structured state maintenance is a credible component of longer-horizon execution. Its value ultimately needs to be assessed through verified completion, preserved correctness, and the resources consumed to achieve both.

Cognaptus: Automate the Present, Incubate the Future.


  1. Chuan Zou and Yujun Zhou and Xiaonan Luo and Xiangliang Zhang (2026). StateHarness: Maintaining Evolving Project State for Long-Horizon Coding Agents. arXiv:2610.17616. https://arxiv.org/abs/2610.17616 ↩︎