Move the Goalposts on Purpose
TL;DR for operators A fixed rubric is a depreciating training asset. Early in reinforcement learning, it may be too demanding to distinguish one weak answer from another. Later, once the model learns to satisfy it, the same rubric becomes too easy. The score survives; the information content does not. EvoRubrics trains the evaluator alongside the model.1 A Policy LLM produces candidate answers, a Rubric Generator produces candidate evaluation criteria, and an external judge scores every answer against every rubric. The policy is rewarded for satisfying the evolving criteria. The rubric generator is rewarded for producing criteria that separate stronger from weaker answers, cover different dimensions, remain anchored to desired preferences, and help the policy revise its responses. ...