Safety

TL;DR for operators Production monitoring often creates an unattractive choice. A small probe is cheap enough to run on every request but may compress away evidence that occurred earlier in a sequence. A stronger position-aware classifier preserves more information but adds computation to an inference path that is already expensive. Speculative Probing: LLM Monitoring at Speculative-Decoding Cost1 asks whether part of that cost has already been paid. Some LLM deployments use a smaller auxiliary prediction component to accelerate generation. Speculative Probing reuses that component—its speculative-decoding head—as a frozen feature extractor for monitoring. The base model and draft head remain unchanged; only a few learned task-specific query vectors and a small classifier are trained. ...

The agent finished the task. That is not the same as doing the task. Inbox sorted. Calendar updated. Report generated. Customer record changed. Dashboard refreshed. For a demo, that is usually enough. The screen shows a plausible answer, the final artifact looks tidy, and everyone politely pretends the agent must have followed the correct path because the output did not immediately burst into flames. ...

A customer asks your AI assistant to “find me a better phone contract.” The agent browses comparison sites, selects a cheaper plan, authorizes the switch, cancels the old plan, and arranges payment of the cancellation fee from the user’s bank account. Lovely, in the way a self-driving forklift is lovely: impressive until it nudges the wrong shelf. ...