The Safety Probe Was Only Part of the Gain

TL;DR for operators

The central finding of Jonathan F. Jao and Andrew Kyle Lampinen’s Value Probes: Learned Value Functions for Beam-Search Red Teaming¹ is that automated safety auditing benefits from two separable improvements: structured exploration of conversations and learned estimates of which conversations deserve further exploration.

On Gemma 3 4B, beam search with randomly selected surviving branches reached a mean peak behavior-judge score of 0.74, compared with 0.51 for token-matched Petri auditing. Adding a Max Value Probe raised the score to 0.85. Similar patterns appeared at 12B and 27B.

For teams auditing internally accessible models, this suggests evaluating search infrastructure before investing in more sophisticated ranking systems. Activation-based probes can subsequently improve branch selection while avoiding expensive prospective LLM-judge calls. Audit histories from smaller models may also help prepare evaluations of newer models.

These gains concern the highest judge score observed during an audit, not the percentage of successful attacks or the completeness of a model’s safety assessment.

Better exploration can reveal failures without better ranking

A multi-turn safety audit can miss a concerning behavior simply because it abandons the relevant conversation too early. A model may respond appropriately for several turns before a later exchange exposes sycophancy, deception, or another targeted behavior.

With limited evaluation compute, an auditor must decide which conversations to continue and which to abandon. Better prompts alone do not resolve that allocation problem.

Jao and Lampinen investigate it by separating the procedure for exploring conversations from the signal used to rank candidate branches.

Their principal comparison involves Petri, an existing automated auditing system, and a beam-search procedure that retains several conversation paths at each turn.

The result is revealing: even when the beam-search procedure selects surviving branches uniformly at random, it substantially improves the measured audit outcome.

Source: Figure 2 and Appendix A.1, Table 1. Values are mean peak judge scores on a 0–1 scale, averaged across 18 behaviors with ten replicate audits per behavior. They are not binary attack-success rates.

The uniform-selection comparison is an ablation of learned branch ranking, not random prompting. Claude Sonnet 4.6 still generates the candidate auditor turns; only the selection of surviving branches is random.

The beam-search configuration retains four active branches, proposes two continuations per branch, and explores up to 12 turns. A full audit therefore generates 96 target-model continuations.

Against token-matched Petri, uniform-selection beam search improves the paired mean peak score by 0.24 at 4B and 0.15 at both 12B and 27B. The corresponding 95% confidence intervals exclude zero.

These results establish a substantial contribution from structured exploration under the tested protocol. They do not isolate every difference between the two auditing harnesses, but they show why the full improvement cannot reasonably be attributed to Value Probes.

For evaluation engineers, this changes the investment sequence: the search procedure itself deserves measurement before adding a learned ranking model.

Value Probes predict future audit value, not just present behavior

Structured exploration still leaves a selection problem. At each depth, which of the candidate conversations is most likely to produce a concerning response later?

Value Probes address that decision using information inside the audited model.

The authors formulate auditing as a Markov decision process. A conversation prefix is a state, the auditor’s next message is an action, and the target model’s reply produces the next state. An LLM judge supplies a behavioral reward.

The critical distinction is between a conversation that already displays the targeted behavior and one that may lead to that behavior after further turns.

To learn this difference, the researchers propagate judge scores backward through sampled conversation trees, discounting future rewards by $\gamma=0.9$.

Two training targets are examined:

• Expected Value: average the estimated values of observed child conversations.
• Max Value: retain the highest estimated value among observed children.

For each conversation prefix, the method extracts residual-stream activations from a selected layer of the frozen target model and averages them across tokens. A behavior-specific linear predictor then estimates the discounted value:

Here, $h_t$ represents the pooled model activations and $w$ is the learned probe vector.

The probe is fitted by squared-error regression. It does not change the target model’s parameters or directly steer its behavior. Its prediction determines which branches the search procedure retains.

This learned ranking adds approximately 0.06–0.10 to paired mean peak judge scores over uniform-selection beam search across the three Gemma 3 sizes. All six paired probe-versus-uniform comparisons have 95% confidence intervals excluding zero.

That is the incremental main evidence for Value Probes: their activation-based predictions improve downstream search decisions beyond the benefit of structured exploration.

The choice between Expected and Max backups appears less consequential at larger scales. Max exceeds Expected at 4B by a paired mean of 0.04 ± 0.03, while the differences at 12B and 27B have confidence intervals including zero. This is an ablation of value-target construction, not proof that the two value functions are equivalent.

A linear probe can replace expensive prospective ranking calls

The authors also compare Value Probes with a more computationally demanding branch-ranking method: asking Claude Opus 4.8 to estimate which candidate conversations are likely to produce the targeted behavior.

The resulting peak scores are close. At 4B, Max Value Probes and prospective LLM ranking both score 0.85. At 12B, they score 0.80 and 0.81 respectively. At 27B, Max Value Probes score 0.80 against 0.76 for LLM ranking.

This comparison supports broadly competitive downstream auditing performance, not consistent superiority of probes over the LLM judge.

The resource difference is more distinctive.

Under the tested configuration, the prospective judge evaluates each of 96 candidate continuations three times, requiring 288 additional Claude Opus 4.8 calls per audit. Value Probes avoid those prospective ranking calls by reading activations already produced through target-model execution.

The savings are specific. Auditor-generated proposals, target-model inference, activation extraction, and post-hoc behavioral evaluation still consume resources. The experiment does not provide a complete monetary or wall-clock cost comparison.

For a safety team operating an open-weight or internally hosted model, activation-based ranking may therefore reduce a significant evaluation bottleneck. An API-only model without activation access cannot use this method directly.

Smaller-model audits can prepare larger-model evaluations

A newly trained model may exhibit relatively few concerning behaviors during initial audits. That scarcity makes it difficult to obtain positive examples for training a ranking system directly on the new target.

The paper’s off-policy experiments examine whether historical audits from a smaller model can help.

The researchers retain transcripts and judge scores generated using Gemma 3 4B, then re-encode those fixed transcripts through the larger target model to obtain its own activations. A new target-specific probe is fitted against the source-derived value targets.

This is not a transfer of the original probe weights into another model’s activation space.

On Gemma 3 12B, off-policy Max Value Probes score 0.81 compared with 0.80 for probes trained on the target’s own rollouts. On Gemma 3 27B, both score 0.80. Expected Value Probes similarly match the rounded on-policy results, and paired differences have confidence intervals including zero.

The transfer experiment extends to Gemma 4 31B. Both off-policy variants achieve 0.79, compared with 0.68 for uniform-selection beam search, outperforming it on 15 of 18 behaviors.

These findings suggest that predecessor-model audit histories can support useful evaluation heuristics without first collecting successful target-generated audits.

However, Gemma 4 31B uses a different post-hoc judge from the Gemma 3 experiments. Its scores should be compared within the Gemma 4 panel, not directly across generations.

Moreover, the transferred value targets originate from source-model outcomes. Their effectiveness as ranking signals does not establish that they accurately represent the target model’s own expected future rewards.

Stronger discovery is not a complete safety assessment

Three boundaries govern how these results should enter model assurance practice.

First, the primary outcome is the maximum behavior-judge score encountered during an audit. A higher maximum indicates that the search found a conversation receiving a higher assessment under the chosen rubric. It does not directly measure confirmed harmful outcomes, time to elicitation, or persistence of the concerning behavior.

Second, the learned probe is predictive rather than causally validated. Its ability to rank promising conversations does not establish that its weight vector represents a mechanism controlling the targeted behavior. The paper reports no activation-intervention experiment demonstrating such control.

Third, evaluation quality depends on the judge and the data-generating procedure. Appendix A.2 provides a separate exploratory warning: contrastive probes achieving in-distribution AUCs of 0.94 can generalize poorly across data-construction strategies. Balanced composite training improves cross-strategy performance, but these experiments are distinct from the principal discounted-value regression method. Appendix A.4 also describes metadata-leakage checks without reporting a complete numerical validation of their outcomes.

For organizations deploying internally accessible language models, a defensible evaluation process would therefore compare token-matched search methods, isolate the incremental effect of learned ranking, and assess judge reliability separately. Reporting should preserve behavior-level uncertainty and distinguish observed audit scores from claims about deployment safety.

The tested evidence is confined primarily to 18 Bloom behaviors and Gemma models, with a fixed search configuration. Cross-family generalization and comprehensive vulnerability coverage remain unresolved.

Search quality deserves its own evaluation budget

The paper’s most consequential finding is the decomposition of auditing performance.

Structured beam search substantially improves behavioral discovery before a learned value estimator is introduced. Value Probes then contribute further gains by predicting which unfinished conversations are worth pursuing, with downstream results comparable to expensive prospective LLM ranking in the evaluated setting.

The transfer experiments add a plausible preparation strategy for model developers: reuse predecessor-model audit histories while fitting new probes to the target model’s own representations.

For safety teams, the resulting priority is specific. Evaluate conversation exploration and branch-ranking intelligence as separate components, measure their contributions under comparable budgets, and retain independent scrutiny of the behavioral judge.

Finding a higher-scoring concerning conversation is valuable evidence about the auditing procedure. It is not, by itself, a complete measurement of the model’s safety.

Cognaptus: Automate the Present, Incubate the Future.