The Agent Was Monitored. Could Its Action Be Stopped?

TL;DR for operators

An AI agent can generate detailed monitoring records while still executing an unsafe tool operation. Prevention requires the operation to be held until an authorized safety decision is made, supported by sufficient evidence and an integration capable of enforcing the decision.

Angelo Ferrando’s AgenticRV separates these responsibilities and allows different monitoring methods to supervise the same execution. In controlled tests, combining temporal and content checks prevented all six constructed unsafe requests without blocking 28 safe requests. However, removing task identity permitted an unauthorized report, and removing upstream approval observations blocked legitimate reports.

For enterprise deployments, the central questions are which operations can actually be intercepted, what evidence each monitor receives, and whether authorization remains attached to the correct task. The paper demonstrates an implementable architecture, not a measured production safety rate.

The tool call was visible, but was it controllable?

Consider an AI incident-response assistant preparing to delete affected resources or transmit an external incident report. The organization needs several answers before execution: Was remediation approved? Is that approval still valid? Does it belong to this incident? Does the outbound report contain restricted information?

A monitoring system might record all these actions and identify violations afterward. That record provides accountability, but it cannot reverse an already executed deletion or disclosure.

Ferrando’s One Monitor Does Not Fit All: Formalism-Agnostic Runtime Verification for Agentic AI¹ makes this distinction explicit. Its implementation separates passive observations from operations that can be intercepted before execution (Section 4.1, Table 1).

In LangGraph, node activity, checkpoints, and lifecycle events can provide monitoring evidence. Guarded native tools, by contrast, can be held before dispatch. MCP and A2A operations can also support preventive decisions, but only when the relevant interposing integration is installed.

This distinction determines whether a safeguard can merely identify a violation or prevent its effect.

AgenticRV separates observation, judgment, and enforcement

AgenticRV introduces a common runtime architecture without requiring every monitor to use the same specification language or internal representation.

Its execution pathway has four responsibilities:

1. Capture execution events. Framework-specific adapters translate activity from LangGraph, MCP, and A2A into canonical events.
2. Provide monitor-specific evidence. Each monitor receives the selected event fields and abstractions needed for its property.
3. Normalize monitoring judgments. Different native verdicts are translated into common control-facing responses.
4. Apply an available intervention. A runtime policy selects a supported reaction, such as allowing or blocking an operation, at an installed decision boundary.

For example, a temporal monitor may examine whether approval and a successful checkpoint occurred recently enough to authorize remediation. A programmatic monitor may separately inspect an outbound report’s content and destination.

Neither monitor needs to adopt the other’s representation. The shared controller nevertheless needs compatible responses to decide whether execution proceeds.

This architecture extends earlier formalism-agnostic monitoring work rather than introducing modular monitoring or runtime enforcement for the first time. Its distinctive contribution is making the observation and control contracts explicit across agent execution boundaries.

Three conditions determine whether monitoring can protect an action

The paper formalizes three requirements that are often treated as implementation details.

First, the observations must be sufficient for the property. If two execution histories look identical to a monitor but require different safety decisions, no exact decision procedure using only those observations can distinguish them. A monitor checking authorization cannot compensate for missing approval history through more sophisticated logic alone.

Second, monitor replacement must preserve the control-relevant response. Different monitoring engines can be substituted without changing the runtime’s decision when the event, deterministic policy, adapter capabilities, and normalized responses—including relevant control metadata—remain equivalent. Sharing an interface does not guarantee equivalent judgments.

Third, prevention requires complete and faithful intervention. The formal safety result assumes every relevant effect crosses an installed synchronous decision boundary, decisions are serialized, unsafe candidates are rejected, and adapters actually prevent rejected effects.

The distinction between attempted operations and completed effects also matters. A rejected remediation request remains available in monitoring history but does not count as completed remediation. A subsequent properly authorized request can therefore proceed without erasing the rejected attempt.

These are conditional guarantees for visible, controlled effects. They do not establish the correctness of an LLM’s internal reasoning.

Two monitors prevented failures that either one missed

The main controlled ablation evaluates seven constructed LangGraph incident-response paths under four monitoring configurations (Section 5.1, Table 2).

Each configuration encounters the same 34 tool requests: six labeled unsafe and 28 safe.

The independent effect ledger confirms whether prohibited operations actually executed, rather than counting only monitor warnings.

The two monitors address different failure mechanisms. The temporal monitor examines approval and checkpoint history, with illustrative freshness limits of six and three logical monitor steps. The programmatic monitor inspects selected report content and destinations.

Operating alone, the temporal monitor misses an unauthorized destination and an incorrect secret classification. The report monitor misses missing or stale approval. Joint monitoring covers both sets of requirements in these constructed paths.

The result supports complementary monitoring for heterogeneous requirements. It does not establish that adding arbitrary monitors always improves protection, or that these two checks cover every unsafe action.

Approval evidence must survive the journey between systems

A separate cross-boundary ablation tests authorization when approval evidence originates in LangGraph and reporting occurs through MCP (Section 5.2, Table 3).

Both monitors remain active. The experiment changes which events the temporal monitor sees and whether incident identities remain distinct.

Erasing incident scope lets approval associated with one incident authorize a report for another. Restricting observations to MCP events creates the opposite problem: legitimate reports are rejected because the monitor cannot see their upstream approvals.

This exposes two distinct engineering defects. Mixing task histories can produce false authorization; withholding relevant history can produce false rejection.

For a multi-agent platform, preserving a task identifier in an event envelope is insufficient unless monitoring state and authorization checks retain the corresponding scope. Approval must also come from trusted completion evidence, not merely a claim embedded in tool arguments.

Implementation tests support the design, within defined limits

The paper provides several additional validation layers, each answering a different question.

Differential testing compares the Reelay temporal monitor against an independent Python evaluator. Across 1,554 enumerated short traces, the experiment reports zero mismatches in 5,910 verdict comparisons. Another 35 targeted cases check approval and checkpoint freshness boundaries.

The selected repository suite reports 179 passing integration tests. MCP and A2A examples exercise interception before controlled execution or dispatch. Four executions using llama3.2:3b through Ollama illustrate one safe path and three paths involving blocked unsafe effects.

Together, these results strengthen confidence in the tested interfaces, temporal implementation, and adapter behavior. The finite comparisons do not prove correctness for unbounded histories, and four LLM executions are illustrative observations rather than an estimate of real-world effectiveness.

Enforcement introduces costs before the monitor does substantial work

The performance microbenchmarks distinguish framework overhead from substantive safety-check computation (Section 5.5, Table 4).

For a minimal LangGraph invocation, mean settled execution time increases from 222.905 microseconds natively to 919.013 microseconds with proxy instrumentation and no monitor. Adding one immediate monitor raises it to 1,332.660 microseconds.

In a separate guarded-tool benchmark, increasing the number of immediate monitors from zero to eight increases mean operation time from 297.919 to 1,166.758 microseconds.

Payload projection and auditing also add measurable costs. These are independent benchmark groups, not cumulative stages of one measured production pipeline.

The measurements use repeated processes and bootstrap confidence intervals, but exclude substantive Reelay computation, remote monitoring, network latency, concurrency, and cold starts. Their value is in identifying fixed framework costs, not forecasting end-to-end enterprise latency.

What enterprises need to establish before relying on a gate

Cognaptus inference: An enterprise deploying agents with destructive tools, privileged record modifications, or outbound reporting should define enforceable operations before selecting monitoring products.

For each consequential operation, the engineering specification should identify the intercepting adapter, required trusted evidence, task scope, monitor responsibilities, supported intervention, and behavior when monitoring fails.

This also requires deciding which histories each monitor must retain. Limiting monitor inputs can reduce unnecessary data exposure, but removing information required for a property destroys the possibility of exact monitoring. Projection does not, by itself, sandbox untrusted monitoring code.

Several deployment risks remain outside the demonstrated guarantees. Unwrapped clients and remote-agent internals may bypass control. Concurrent operations need additional atomic enforcement or property-specific reasoning. Distributed event order, evidence authenticity, adapter failures, and fail-open policies require explicit treatment.

The controlled results also provide no head-to-head evidence that AgenticRV outperforms other agent guardrail systems in deployed environments.

The safety boundary is an engineering decision

AgenticRV shows that heterogeneous monitors can be coordinated through a reusable runtime interface and that complementary checks can prevent unsafe effects in carefully controlled workflows.

Its strongest lesson concerns the contract between evidence and execution. A monitor needs enough information to reach the right judgment, while the runtime needs authority to enforce that judgment before the effect occurs.

For organizations granting AI agents consequential tool access, safety assurance begins by establishing those contracts at every relevant execution boundary. The paper demonstrates how to implement and test them; production reliability remains a separate engineering and validation obligation.

Cognaptus: Automate the Present, Incubate the Future.