Fixed Priorities, Moving Rewards: What ReCAST Changes in Diffusion Training

TL;DR for operators

An image-generation product may need to follow prompts, produce attractive images, and render specified text correctly. Its team can assign a fixed importance to each objective, but that does not establish when each objective’s feedback should influence training. ReCAST separates these decisions: it preserves specified reward budgets while reallocating their influence across diffusion denoising steps. In experiments on Stable Diffusion 3.5-Medium, this improves aggregate training rewards for text rendering but not for compositional generation; independent reward models nevertheless favor ReCAST on average in both settings. For training teams, the opportunity is more selective reward allocation without surrendering control of objective priorities. The tradeoff is expensive, approximate temporal profiling whose reliability beyond the tested configuration remains uncertain.

A fixed reward weight cannot describe when feedback becomes informative

In multi-objective image generation, reward weights normally express the relative importance of different outcomes. Conventional training applies those relative weights consistently across denoising timesteps.

That consistency has a limitation. A diffusion model progressively converts noisy representations into images, and the information available for evaluating different visual properties changes along the process. A signal concerning composition may become informative at a different stage from one concerning visual appearance or text accuracy.

The evidence indicates that these differences can be substantial. In the investigated setting, estimated reward-specific discriminability curves differ across timesteps. Under uniform reward budgets, their combined normalized gain demand varies by a factor of 5.4 across timesteps 2 through T. Excluding the GenEval reward still leaves a 4.5-fold variation.

These measurements are estimated informativeness profiles, not observed improvements in reward scores at each timestep. They nevertheless expose a question static weighting cannot answer: how should a fixed objective budget be distributed when the signal associated with that objective varies over time?

Chen and colleagues address this question in ReCAST: Reward Credit Assignment across Timesteps for Online Diffusion Reinforcement.¹ Their proposal changes the temporal allocation of feedback inside an existing diffusion reinforcement-learning framework, DiffusionNFT. It does not introduce a new image sampler or replace the underlying reinforcement objective.

ReCAST changes reward timing while preserving objective budgets

The method begins with a reward-by-timestep allocation matrix, $W$. Its entry $W_{i,t}$ specifies how much of reward $i$ is assigned to denoising timestep $t$.

Two constraints govern this allocation:

Here, $\lambda_i$ is the normalized, externally specified budget for reward $i$, and $T$ is the number of denoising steps.

The first constraint preserves the product owner’s relative priorities. The second ensures that the combined reward allocation remains equal across timesteps.

Consequently, a reward can receive greater influence during particularly informative steps only by receiving less elsewhere. ReCAST cannot independently amplify every reward at every timestep its estimator favors.

The remaining problem is determining which timesteps deserve greater influence for each reward.

Each reward implicitly favors a different distribution of generated images. ReCAST represents this through a reward-induced positive policy: a distribution that assigns more probability to images receiving higher values under that reward.

When noise is added to an image, the difference between its reward-favored distribution and the current model’s distribution becomes harder to distinguish.

The researchers quantify this separation using Rényi divergence. The incremental difference between consecutive noise levels is the reward’s discriminability gain—the additional distributional distinction recovered during that denoising transition.

Under the paper’s shared forward-diffusion dynamics and regularity assumptions, its dissipation theorem establishes that the population divergence cannot increase as noise increases. The corresponding denoising-direction gains are therefore nonnegative and telescope to total clean-image discriminability.

This gives temporal reward assignment a mathematical basis: the allocation can respond to newly distinguishable reward information instead of relying on a single schedule for all objectives.

The estimates are approximations, not exact divergences

Directly sampling from each reward-induced positive policy is difficult. ReCAST therefore uses GPT Image 1.5 as a common external proposal generator, introduces noise into generated images, and evaluates conditional rollouts from the current model.

These rollouts produce surrogate temporal gain estimates. Their shape is normalized separately for each reward and exponentiated into a reward-timestep affinity matrix.

However, independently assigning each reward according to its preferred timesteps need not satisfy the shared timestep constraint.

ReCAST resolves this using Sinkhorn projection, an entropy-regularized matrix-scaling procedure. It finds an allocation close to the estimated affinities while enforcing both the reward-budget totals and the common timestep totals.

The result is a coordinated allocation across objectives, not a collection of independently optimized schedules. Importantly, the theorem concerns exact population divergences; the external-generator estimates are biased proxies whose accuracy depends on additional assumptions.

OCR rewards improve; GenEval rewards largely do not

The researchers fine-tune Stable Diffusion 3.5-Medium using LoRA adapters and DiffusionNFT, with 25 denoising steps.

A common warmup trains three rewards: ClipScore, HPSv2, and PickScore. Two subsequent configurations add either an OCR text-rendering reward or a GenEval compositional reward.

Each configuration is evaluated across five reward-budget settings and three independent seeds. ReCAST and static weighting use matched training conditions, providing 30 budget-seed comparisons across the two settings.

The training results are not uniformly favorable.

Source: Section 4.2, Table 1. Scores are aggregate native training-reward values, averaged over the reported budget and seed configurations.

The OCR improvement is 0.053, approximately 1.85% relative to static weighting. All four optimized rewards also improve on average in that setting, although the individual matched-pair results differ. PickScore improves in all 15 comparisons; OCR itself improves in nine.

GenEval presents a different outcome. Its aggregate training reward changes by just -0.003, and individual optimized reward differences are small relative to seed-level variation.

The more consequential finding emerges when the evaluation moves beyond the rewards used for optimization.

Source: Section 4.3, Table 2. Each metric retains its own scale; differences must not be pooled into a common effect size.

All four held-out judges report higher average scores for ReCAST in both settings. OCR matched-pair wins range from 11 to 13 out of 15, while GenEval wins range from nine to ten.

This indicates that changing reward timing can affect evaluations beyond the explicitly optimized reward functions, even when the optimized aggregate itself shows little improvement.

An additional automated multimodal evaluation favors ReCAST in four of five OCR budget configurations. Its position-debiased overall win rate averages 57.18% across the reported comparisons, using a fixed 1,000-prompt subset and reversed presentation orders.

That is evidence from an independent automated judge, not a direct human preference study. The source also reports no formal statistical significance test for the aggregate comparisons.

The ablation separates adaptive timing from shared-budget coordination

A central design question is whether ReCAST benefits mainly from discovering reward-specific timing or from enforcing a coordinated timestep budget.

The researchers test this by removing Sinkhorn projection and retaining row-normalized reward-specific Rényi weighting.

In the reported single-seed OCR ablation, reward-specific weighting without Sinkhorn improves aggregate reward over static weighting in four of five budget configurations. Full ReCAST then improves over the no-Sinkhorn version in four of five.

The authors’ aggregate decomposition assigns approximately 0.068 of a 0.099 improvement to reward-specific adaptation and a further 0.031 to Sinkhorn projection.

This supports two contributing mechanisms: estimating when rewards are informative and coordinating their allocation under shared constraints. But the decomposition comes from one training seed, not the full three-seed evaluation. It should not be treated as a stable percentage attribution across configurations.

The equal aggregate timestep budget is also an architectural choice. The experiments support its contribution within this comparison, not its universal optimality.

The deployment decision depends on profiling cost and schedule stability

For teams developing image-generation products, Cognaptus draws a specific operational inference: objective governance and training-stage optimization can be separated.

Product owners can specify the relative importance of prompt fidelity, appearance, composition, and text accuracy. Training engineers can then estimate where each reward provides useful feedback, producing an inspectable allocation matrix whose totals remain consistent with those priorities.

The research does not establish that this division automatically improves every objective. It suggests a training-design option worth evaluating when several reward models are optimized together.

Three boundaries determine whether the option is economical and reliable.

First, the estimator is computationally expensive. The reported profiling cost is approximately 15 GPU-hours for a 10-step curve and about six times as much for a 25-step training-stage curve, excluding external API latency. Reusing a profile across multiple reward budgets could amortize the cost, but the paper does not establish a production break-even point.

Second, proposal quality matters. Replacing GPT Image 1.5 with Nano Banana Pro produces correlated temporal profiles, with Pearson correlations of 0.847–0.919 across four OCR-stage rewards. This sensitivity test suggests some stability across the two proposals. It does not show that either accurately approximates every reward-specific positive policy.

Third, the useful schedule may change during training. ReCAST constructs its gain curves before each stage and reuses them. When the authors re-estimate the common-proposal quantities after OCR fine-tuning, their combined estimated total declines from 3.241 to 0.113. These are surrogate measurements rather than exact remaining divergences, but the change raises a concrete question about when temporal profiles should be refreshed.

Evidence also remains limited to one base architecture, two four-reward training configurations, and three seeds per budget. Generalization to larger objective collections or other generative modalities is untested.

Reward allocation is a separate training decision

ReCAST’s contribution is not a new way to decide which image-quality objectives matter most. It supplies a mechanism for distributing already specified priorities across denoising steps, using reward-specific distributional information and explicit allocation constraints.

The strongest empirical case is the OCR configuration, where both optimized and held-out evaluations improve. GenEval provides a more qualified result: optimized reward parity alongside favorable average held-out scores.

For model-training teams, the appropriate next decision is therefore comparative and measurable. A ReCAST-style implementation should be justified by independent quality gains relative to static weighting, after accounting for profiling costs, estimator stability, and the need to update schedules as the model changes.

The broader insight is that a reward’s importance and the timing of its influence are different optimization variables. ReCAST demonstrates one constrained way to manage both without allowing training mechanics to rewrite the objectives themselves.

Cognaptus: Automate the Present, Incubate the Future.