The Safety Probe Was Only Part of the Gain
TL;DR for operators The central finding of Jonathan F. Jao and Andrew Kyle Lampinen’s Value Probes: Learned Value Functions for Beam-Search Red Teaming1 is that automated safety auditing benefits from two separable improvements: structured exploration of conversations and learned estimates of which conversations deserve further exploration. On Gemma 3 4B, beam search with randomly selected surviving branches reached a mean peak behavior-judge score of 0.74, compared with 0.51 for token-matched Petri auditing. Adding a Max Value Probe raised the score to 0.85. Similar patterns appeared at 12B and 27B. ...