Move the Goalposts on Purpose

TL;DR for operators

A fixed rubric is a depreciating training asset. Early in reinforcement learning, it may be too demanding to distinguish one weak answer from another. Later, once the model learns to satisfy it, the same rubric becomes too easy. The score survives; the information content does not.

EvoRubrics trains the evaluator alongside the model.¹ A Policy LLM produces candidate answers, a Rubric Generator produces candidate evaluation criteria, and an external judge scores every answer against every rubric. The policy is rewarded for satisfying the evolving criteria. The rubric generator is rewarded for producing criteria that separate stronger from weaker answers, cover different dimensions, remain anchored to desired preferences, and help the policy revise its responses.

The practical proposition is larger than “automate rubric writing.” It is that evaluation should operate at the same temporal granularity as optimization. Rather than updating a checklist once per project, epoch, or model release, EvoRubrics updates its evaluator inside every training step. This creates an automatic curriculum: as the policy closes obvious gaps, the evaluator must find subtler ones.

The strongest evidence is in-domain. On the Qwen3-4B backbone, EvoRubrics raises the HealthBench Hard normalized ratio from 22.48 for the base model and 27.75 for fixed golden-rubric RL to 36.64. It also performs well on an unseen medical benchmark. Out-of-domain results are more mixed: FollowBench improves, while MT-Bench is largely preserved rather than decisively beaten. This is a useful result, but it is not evidence that an adaptive medical evaluator has discovered universal intelligence. One must occasionally disappoint the branding department.

The evolved evaluator can also be reused. It modestly improves pairwise preference ranking on RubricBench and improves HealthBench responses when its criteria are inserted as inference-time guidance. That creates a plausible route from training infrastructure to a reusable evaluation service.

There are material boundaries. The experiments use medical-domain training, Qwen3 models no larger than 8B, an external DeepSeek-V3.2 judge, supervised warm-start data generated by DeepSeek-R1, and only one run per experiment. Training takes 60.5 hours on eight A100-80GB GPUs, versus 33.9 hours for fixed golden-rubric RL—a 78% wall-clock premium. The self-supervised variant removes reference-rubric supervision, not the external judge or the rest of the training machinery.

For businesses, the relevant question is therefore not whether dynamic rubrics are “better.” It is whether the cost of a stale evaluator—reward saturation, specification gaming, missed edge cases, and repeated human redesign—exceeds the cost of operating an evaluator that learns.

Static evaluation becomes less useful precisely when the model improves

Most operational evaluation systems assume that the target remains stationary.

A support assistant should be accurate, polite, complete, and compliant. A medical-answering model should identify urgent symptoms, avoid unsafe recommendations, explain uncertainty, and advise professional care when appropriate. These requirements can be represented as a rubric: a list of desirable and undesirable properties, each assigned a positive or negative weight.

For an answer (a) and rubric (R), the paper defines a weighted score of the form

where (d_k) is a criterion, (w_k) is its weight, and (J) is a judge deciding whether the answer meets that criterion. The score is then normalized by the maximum positive score available under that rubric.

This decomposition is attractive because it turns an opaque scalar reward into visible components. A model can receive credit for identifying an emergency warning sign and a penalty for suggesting unsafe self-treatment. Operators can inspect the criteria instead of staring at a reward of 0.63 and pretending it contains a theory of quality.

The problem appears over time.

At the beginning of training, a sophisticated rubric may assign nearly identical low scores to every response. The model is too weak for the evaluator to detect useful differences. Later, the policy learns the obvious requirements. Most answers satisfy most criteria, scores cluster near the top, and the rubric again stops distinguishing useful variation.

The same instrument can therefore fail at both ends:

This is not unique to machine learning. Once people know the performance metric, they reorganize around it. Models merely do so with fewer meetings.

The accepted misconception to avoid is that EvoRubrics periodically asks a model to refresh the checklist. That would be dynamic administration, not co-evolution. The paper’s central mechanism is more tightly coupled: the policy and evaluator generate, score, and update against one another within the same training step.

EvoRubrics puts the policy and evaluator in one feedback loop

EvoRubrics begins with a shared backbone model and gives it two separate LoRA adapters.

One adapter creates the Policy LLM. The other creates the Rubric Generator. Activating the policy adapter makes the model answer a query; activating the rubric adapter makes it generate between nine and eleven weighted evaluation criteria. With neither adapter active, the underlying backbone serves as the reference model used for KL regularization.

For each query batch, training proceeds through four phases:

Query batch
    |
    +--> Policy adapter generates candidate answers
    |
    +--> Rubric adapter generates candidate rubric sets
                    |
                    v
        External judge scores every
        answer-rubric combination
                    |
                    v
             Score matrix
              /        \
             /          \
    Policy rewards    Rubric rewards
          |                |
          v                v
    Update policy     Update evaluator
          \                /
           \______________/
          Next training step

Suppose the policy generates (M) candidate answers and the evaluator generates (N) rubric sets. The judge evaluates all (M \times N) combinations.

Each answer’s policy reward is its average normalized score across the generated rubrics. This reduces dependence on any one eccentric checklist. The policy adapter is then updated using Group Relative Policy Optimization, comparing candidate responses generated for the same prompt.

The evaluator receives a different reward. Its task is not to make the scores high. It is to produce criteria that remain useful for improving the policy.

That distinction matters. A cooperative evaluator could reward everything and become pleasantly useless. A purely adversarial evaluator could invent impossible, irrelevant, or pathological standards. EvoRubrics instead tries to create bounded adversarial pressure: difficult enough to expose remaining weaknesses, constrained enough to keep “better” connected to human-defined quality.

Because both adapters are updated on the same query batch, the next training step begins with a slightly stronger policy and a slightly altered evaluator. The curriculum is not designed in advance. It emerges from the evaluator’s need to keep separating the policy’s latest outputs.

Four rewards stop the evaluator from becoming an enthusiastic nuisance

The evaluator is optimized using four equally weighted reward components: discrimination, diversity, alignment, and constructiveness.

These are not four decorative synonyms for “quality.” Each controls a different failure mode.

Discrimination keeps the reward informative

For each generated rubric, EvoRubrics examines how widely its scores vary across the policy’s candidate answers. A rubric that assigns every answer 0.72 has almost no value for group-relative optimization. A rubric that reliably separates stronger and weaker responses provides a usable gradient.

This is the core adversarial component. As the model learns to satisfy broad criteria, the evaluator must generate finer distinctions to maintain score variance. Yesterday’s difficult requirement becomes today’s baseline.

There is an obvious risk: variance alone does not imply correctness. An evaluator could separate answers using arbitrary stylistic quirks. The remaining rewards are therefore not optional philosophical accessories. They are the restraints on the discrimination objective.

Diversity broadens the evaluator’s field of view

The evaluator embeds the descriptions of its generated criteria and is rewarded when those embeddings are semantically distant.

The intention is straightforward. A medical response should not be evaluated using ten near-duplicate criteria about politeness while factual omissions, emergency escalation, contraindications, and uncertainty go unexamined.

Embedding distance is only a proxy for conceptual coverage. Semantically different criteria can still be irrelevant, and related criteria may both be necessary. Nevertheless, the reward gives the evaluator an explicit reason not to collapse onto whichever dimension is easiest to detect.

Alignment keeps difficulty connected to human preferences

Generated rubrics are compared with pre-constructed reference rubrics using an external judge. High semantic alignment earns a higher reward.

This anchor is crucial because policy and evaluator share the same backbone. Shared representations make interaction easier, but they also create correlated failure modes. If both components adopt the same strange preference, their agreement does not make the preference desirable.

Alignment therefore acts as a governor. The evaluator may raise the bar, but it is not supposed to relocate the playing field.

Constructiveness tests whether criticism can improve an answer

For each generated rubric, the policy revises a baseline answer using the rubric as feedback. The revised response is then evaluated against the reference rubric. If the revision improves, the generated rubric receives a higher constructiveness reward.

This is a more demanding test than asking whether a criterion sounds sensible. The rubric must cause an observable improvement under an independently anchored evaluation.

Operationally, this is one of the paper’s most useful ideas. Evaluation criteria should not merely classify outputs; they should increase the probability that a correction process produces a better output. A critic that cannot support repair may still be useful for auditing, but it is a weaker training instrument.

The four rewards together produce a controlled tension:

The paper weights the four components equally. It does not establish that equal weighting is optimal across domains. A regulated workflow may need a heavier alignment term; a creative system may value broader diversity. The design is best interpreted as a workable decomposition, not a universal constitution.

Dual LoRA makes two roles affordable, not free

Training a full policy model and a full evaluator model would be expensive and would allow their capabilities to diverge. EvoRubrics instead places two LoRA adapters on the same Qwen3 backbone.

The arrangement offers three advantages.

First, both roles inherit the same underlying knowledge. The evaluator is not attempting to diagnose subtle weaknesses in a policy operating at an entirely different capability level.

Second, policy and evaluation can specialize independently. Gradients update only the active adapter, so the rubric generator does not simply become the policy with a different prompt.

Third, adapter switching avoids maintaining two complete models in memory. The paper reports that the adapters account for about 0.5% of the base model’s parameters and that switching plus weight synchronization takes roughly two seconds.

There is a strategic trade-off hiding inside this efficiency.

A shared backbone can improve compatibility because the evaluator and policy represent the domain similarly. It can also increase correlated blindness. Two employees trained in the same organization may communicate efficiently while sharing the same institutional delusion. Architectural convenience is not epistemic independence.

EvoRubrics retains an external DeepSeek-V3.2 judge for scoring. That judge provides some separation, but it also means the system has not eliminated dependency on a stronger external model. It has changed where that dependency enters.

The main evidence is a strong medical-domain gain without a general collapse

The experiments train Qwen3-4B and Qwen3-8B models on 4,000 standard-difficulty HealthBench examples. Evaluation uses 1,000 HealthBench Hard cases, the unseen medical RaR-Medicine benchmark, and two out-of-domain benchmarks: MT-Bench and FollowBench.

All reinforcement-learning methods begin from the same supervised fine-tuning checkpoint. The comparison includes fixed expert-written rubrics, rubric-scaffolded RL, and an online dynamic-rubric baseline that asks an external model to extract and accumulate criteria during training.

The clearest result appears on Qwen3-4B:

Three interpretations follow.

First, the in-domain improvement is substantial. EvoRubrics raises the HealthBench Hard ratio by 8.89 points over fixed golden-rubric RL and by 14.16 points over the base model. Its absolute HealthBench score reaches 20.80, compared with 15.82 for GoldenRubrics and 12.62 for the base model.

The Qwen3-4B EvoRubrics model also exceeds the reported DeepSeek-V4-Flash prompt baseline on HealthBench Hard: 20.80 versus 18.04 in raw score and 36.64 versus 33.03 in normalized ratio. This is evidence that domain-specific post-training can outperform a larger prompted model on the target benchmark. It is not evidence that 4B models have rendered scale obsolete. Sadly, one benchmark victory still does not constitute a procurement strategy.

Second, fixed rubric optimization can damage broader performance. On the 4B backbone, GoldenRubrics falls below the base model on RaR-Medicine, both rounds of MT-Bench, and FollowBench. Optimizing against a static medical rubric does not simply improve the target domain and leave everything else untouched.

EvoRubrics avoids much of that damage. It improves the unseen medical benchmark and FollowBench while roughly preserving MT-Bench. Its MT-Bench Round-1 score of 7.74 remains slightly below the base model’s 7.82, while Round 2 rises from 6.82 to 7.11. The appropriate claim is therefore resilience, not universal out-of-domain superiority.

Third, the 8B results tell a similar but less tidy story. EvoRubrics reaches a HealthBench Hard ratio of 42.27, above 38.33 for GoldenRubrics and 25.92 for the base model. It also improves FollowBench. On MT-Bench, however, it does not dominate the supervised model: its Round-2 score is 7.05 versus 7.43 for SFT.

The mechanism appears most valuable where evaluation criteria are rich, domain-specific, and capable of becoming stale. On broad conversational benchmarks, the evidence is closer to “does not catastrophically overfit” than “discovers a general-purpose reward function.”

The evaluator becomes a reusable asset, although the gains are modest

The paper evaluates the trained Rubric Generator separately from the policy.

In the first transfer test, generated rubrics are used as a reward model on RubricBench, which contains 1,147 human-labelled pairwise comparisons across instruction following, STEM, code, safety, and open-ended chat. The evaluator generates criteria for each query, and a judge uses those criteria to select the better response.

The 4B evaluator achieves 51.90% overall agreement, compared with 48.90% for the untrained base model and 50.70% after supervised fine-tuning. The 8B evaluator reaches 52.40%, slightly above the reported Qwen3-32B prompt baseline of 51.53%.

These are real improvements, but not enormous ones. The result supports the proposition that the evaluator learns something transferable beyond its immediate training loop. It does not show that it has become a highly accurate standalone judge.

In the second transfer test, the evaluator generates rubrics that are prepended to the user prompt as optional guidance. On HealthBench, this raises the 4B model’s ratio from 26.33 to 27.87 and the 8B model’s ratio from 43.56 to 44.83.

Again, the effect is modest. Its strategic relevance comes from reuse rather than magnitude. The same evaluator can potentially serve three roles:

1. generate rewards during training;
2. rank candidate outputs after training;
3. produce query-specific checklists for inference-time self-correction.

That is a more attractive infrastructure story than a component discarded once training ends. It also creates a governance problem: when the same evaluator influences training, selection, and inference, its blind spots can propagate through the entire system.

The ablations test the mechanism, not a second thesis

The ablation study removes each evaluator reward component on the Qwen3-4B setup. Its purpose is to determine whether the four-part reward design is doing useful work, not to establish another independent performance claim.

The full system achieves ratios of 36.64 on HealthBench and 60.50 on RaR-Medicine. Removing constructiveness reduces them to 31.86 and 46.64. Removing diversity produces 31.75 and 47.58. Those are the largest in-domain declines.

Removing discrimination lowers the four reported metrics—HealthBench, RaR-Medicine, MT-Bench Round 1, and FollowBench HSR—supporting the authors’ interpretation that score separation is the foundational signal. A rubric that cannot distinguish candidate answers leaves GRPO with little to optimize.

Alignment also matters, although its in-domain decline is smaller: HealthBench falls from 36.64 to 34.45. This does not make alignment dispensable. Its primary role is directional control, which may not be fully captured by the target-domain score.

The most informative row removes all four evaluator rewards, effectively freezing the Rubric Generator after initialization. This frozen version still reaches a HealthBench ratio of 29.14, above GoldenRubrics at 27.75, but well below the complete system at 36.64.

Two conclusions can be separated.

The result directly supports the claim that continuous evaluator learning contributes beyond merely starting with generated rubrics.

The authors additionally suggest that rubrics produced by the shared backbone may be easier for the policy to optimize than externally authored criteria because they inhabit a compatible representation space. That is plausible, but the ablation does not isolate representation compatibility from other differences in rubric wording, distribution, or initialization. It is a mechanism hypothesis, not yet a settled property of shared-backbone evaluators.

Self-supervision creates a curriculum, then reveals what the anchor was doing

The paper’s exploratory extension removes the human-authored reference rubrics from evaluator training.

Discrimination and diversity remain. Alignment is removed. Constructiveness becomes self-referential: the policy revises an answer using a generated rubric, and the revision is scored using that same rubric.

This setup is described as fully self-supervised because it does not use external rubric supervision. The term should not be read more broadly than that. The experiment still uses the external DeepSeek-V3.2 judge, begins from a supervised checkpoint trained on DeepSeek-R1 responses, and depends on the surrounding reinforcement-learning pipeline.

On HealthBench, the self-supervised 4B policy performs surprisingly well:

This result supports an important theoretical point: interaction between a policy and an adaptive evaluator can generate a useful curriculum even without a human-written rubric anchor.

The broader benchmark pattern is less flattering.

The self-supervised model falls below the base model on RaR-Medicine, both rounds of MT-Bench, and FollowBench. It performs around or somewhat above GoldenRubrics on several of those measures, but it does not preserve the broader transfer profile of the anchored system.

The evaluator shows the same narrowing. Its overall RubricBench accuracy rises from 48.90 for the base model to 50.96, but below the anchored evaluator’s 51.90. Its safety accuracy jumps to 53.75, far above both the base and anchored versions, while its performance becomes less balanced across other categories. The paper interprets this as drift toward safety-heavy characteristics learned from the medical domain.

That is the more useful business lesson.

Self-supervision can reduce annotation dependence, but removing the anchor does not remove values. It allows the training domain, judge behaviour, shared model biases, and easiest-to-optimize signals to become the de facto values.

A system without an explicit preference anchor is not neutral. It is merely governed by less visible preferences.

Adaptive evaluation costs more because it performs more evaluation

EvoRubrics reduces memory duplication through dual LoRA adapters, but its training loop remains expensive.

The paper reports the following wall-clock times for 500 training steps on eight NVIDIA A100-80GB GPUs:

The main overhead comes from cross-evaluating answers and rubrics and performing a second GRPO update for the evaluator. Judge calls are dispatched concurrently, preventing wall-clock time from scaling linearly with their raw count. The dual adapters add relatively little GPU cost.

A local judge could reduce external API latency and expense, but that would not eliminate the computation. It may also increase correlated error if the local judge shares the same model family or training data.

The relevant ROI equation is therefore not:

Does adaptive evaluation cost more than static evaluation?

It plainly does in this implementation.

The better question is:

Does the incremental training cost buy more value than repeated expert rubric redesign, failed model iterations, reward hacking, and domain-specific quality defects?

For a low-risk copywriting system, probably not. For a high-volume support platform, regulated workflow, or costly expert domain, the answer may be different.

The business value is continuous diagnosis, not automated checklist production

The paper directly demonstrates a training method. Applying it to business systems requires an interpretive step.

Three use cases appear particularly plausible.

High-dimensional outputs without one correct answer

Rubric co-evolution is most relevant when output quality has multiple interacting dimensions and cannot be verified using an exact answer.

Examples include customer-support responses, compliance explanations, medical information, legal document assistance, research synthesis, and complex agent reports. These tasks already rely on evolving review standards. EvoRubrics offers a way to make part of that evolution endogenous to training.

Models that repeatedly exhaust their test suite

Teams often discover that a benchmark stops providing useful information once the model has been optimized against it. They then commission harder examples, add edge cases, or revise the scoring protocol.

A learned evaluator can perform a version of this red-teaming continuously. It watches the policy’s current responses and seeks criteria that still expose differences. The operational value is cheaper diagnosis of the next weakness, not simply a higher score on the current checklist.

Workflows that can reuse evaluation at inference time

An evaluator that generates query-specific checks can support candidate ranking, response revision, escalation decisions, and audit trails.

This reuse matters because evaluator development otherwise tends to be treated as non-product infrastructure. If the evaluator improves live output quality as well as training, the cost can be allocated across more than one stage of the model lifecycle.

A production version needs an evaluator governor

The paper’s four rewards can be translated into a practical control framework.

A business implementation should not allow the generator and evaluator to co-evolve inside a sealed loop. It needs at least four independent controls.

A stable preference anchor. Some portion of the evaluation standard should remain human-owned and version-controlled. The anchor need not cover every case, but it must define qualities the evaluator is not permitted to optimize away.

An independent holdout judge. The component measuring evaluator quality should not be identical to the policy, rubric generator, or production selector. Otherwise, the system can become internally consistent while externally wrong.

A drift dashboard. Teams should track which criterion categories grow or disappear over training, whether score variance is being created by meaningful distinctions, and whether performance is shifting among domains or user groups.

A retirement test for old criteria. Adaptive evaluation should not merely accumulate more rules. Criteria that no longer discriminate, duplicate other criteria, or induce counterproductive behaviour should be identified explicitly.

The architecture can be understood as a separation of responsibilities:

Human governance
      |
      v
Stable anchor criteria -------> Drift and safety audits
      |                               ^
      v                               |
Adaptive Rubric Generator ---> Independent evaluator
      |                               |
      v                               |
Policy training and revision --------+

This adds bureaucracy to an already complicated training loop. That is unfortunate, but less unfortunate than discovering that the evaluator and policy have spent several million tokens agreeing with each other.

The paper establishes a direction, not a deployment recipe

Several limitations materially affect interpretation.

Training is concentrated in one domain. HealthBench provides 4,000 training examples and the main evaluation set comes from the same dataset family. RaR-Medicine provides a useful unseen medical test, while MT-Bench and FollowBench offer broader checks, but none establishes that co-evolution will behave similarly in legal analysis, finance, creative work, coding agents, or multilingual systems.

The tested models are limited to Qwen3-4B and Qwen3-8B. Larger dense models and mixture-of-experts architectures may change both the benefits and the operational costs. A stronger policy could saturate the evaluator differently; a stronger evaluator could create harsher but less stable feedback.

Each experiment is run once because of GPU and API cost. Without repeated runs, the paper cannot estimate variance or establish how sensitive the results are to rollout sampling, judge behaviour, initialization, or training noise.

The judge is deeply embedded in the system. DeepSeek-V3.2 scores answer-rubric pairs and contributes to alignment and constructiveness calculations. Consequently, improvements may partly reflect better adaptation to that judge’s preferences. A different judge could alter both the curriculum and the measured outcome.

The shared backbone creates potential correlated bias. The authors explicitly note that co-evolving generation and evaluation could amplify spurious preferences, narrow criteria, or reward-model idiosyncrasies. The self-supervised safety skew provides a small empirical warning of this general risk.

Finally, the medical examples are research outputs, not clinically validated systems. The appendix cases show more complete warnings and safer escalation after training, but benchmark improvement is not clinical approval. Deployment would require independent medical validation, monitoring, and regulatory compliance.

Moving the goalposts is sensible when someone still controls the stadium

EvoRubrics makes a convincing case that evaluation criteria should not be treated as static labels attached to a dynamic optimization process.

Its main contribution is the within-step feedback loop. The policy improves against the evaluator’s latest criteria; the evaluator improves by finding distinctions among the policy’s latest answers. The four-part evaluator reward prevents that competition from collapsing into either indiscriminate generosity or arbitrary hostility.

The empirical picture is promising and appropriately uneven. The method produces large gains on the target medical benchmark, useful transfer within medicine, and better instruction-following results without wholesale destruction of broad conversational performance. The evaluator itself becomes modestly reusable. The self-supervised experiment shows that adaptive evaluation can generate learning signals without reference rubrics, while simultaneously demonstrating why preference anchors exist.

For operators, this points toward a change in how evaluation infrastructure is designed.

A rubric should not be considered finished when it has been written. It should be monitored for saturation, challenged by current model outputs, revised when it stops distinguishing meaningful quality, and constrained by standards that remain outside the model’s control.

The goalposts should move because capability moves. They should not move simply because the model has learned to push them.

Cognaptus: Automate the Present, Incubate the Future.