DiffusionOPSD

DiffusionOPSD (On-Policy Self-Distillation in Diffusion Models) is a post-training method for image diffusion models from ByteDance. It converts image-level reward guidance into bounded positive and negative targets for clean-output predictions at sampled low-noise query states, giving explicit intermediate supervision where endpoint rewards alone do not.

The training loop has three steps: a frozen behavior policy generates trajectories and supplies low-noise queries and clean-output anchors; normalized reward ascent and descent build positive and negative targets within a controlled radius; and the trainable policy fits these detached targets under a finite budget, after which an EMA update refreshes the behavior policy. An accelerated variant, DiffusionOPSD*, is also reported.

The project page reports matched experiments on SD3.5-M and the step-distilled Z-Image-Turbo across ten evaluators, with the best final held-out scores in reward-matched settings and fewer training GPU-hours than DiffusionNFT (about 40% fewer on SD3.5-M and 63% fewer on Z-Image-Turbo). It also provides diagnostics that separate target construction from finite-update realization, including a CFG-free training analysis. A YouTube creator mentioned that code was released, but no repository link was found in the provided material. It is a research method, not a packaged product.