Flash GRPO

Flash-GRPO is a research method for aligning video diffusion models with human preferences using Group Relative Policy Optimization (GRPO). It targets the computational cost of GRPO training, which the authors say can take hundreds of GPU days per experiment for a 14B-parameter model. Instead of training over the full sampling trajectory or a sliding window of timesteps, it trains on a single sampled timestep per prompt.

The method has two components. Iso-temporal grouping has each prompt switch from ODE to SDE sampling at one sampled timestep for exploration and gradient computation, while other timesteps use deterministic ODE. Rollouts in a group share this timestep but differ in initial noise, which separates policy-induced variance from timestep-induced variance. Temporal gradient rectification normalizes by the time-dependent scaling factor λ(t) from SDE discretization, so gradient magnitudes are consistent across timesteps.

The project page reports experiments on Wan2.1 models from 1.3B to 14B parameters, with claims of faster training, stable optimization and alignment quality that matches or exceeds full-trajectory training under low compute budgets. A YouTube creator summary says code has been released, but no repository link was available in the provided material. It is a training technique for researchers, not a standalone product.