Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914569335603200 |
|---|---|
| author | He, Xiaoxuan Fu, Siming Xue, Zeyue Wang, Weijie He, Ruizhe Li, Yuming Yin, Dacheng Dong, Shuai Huang, Haoyang Wang, Hongfa Duan, Nan Zhuang, Bohan |
| author_facet | He, Xiaoxuan Fu, Siming Xue, Zeyue Wang, Weijie He, Ruizhe Li, Yuming Yin, Dacheng Dong, Shuai Huang, Haoyang Wang, Hongfa Duan, Nan Zhuang, Bohan |
| contents | Group Relative Policy Optimization has emerged as essential for aligning video diffusion models with human preferences, but faces a critical computational bottleneck: training a 14B parametered model typically demands hundreds of GPU days per experiment. Existing efficiency methods reduce costs through sliding window subsampling training timesteps, but fundamentally compromise optimization, exhibiting severe instability and failing to reach full trajectory performance. We present Flash-GRPO, a single-step training framework that outperforms full trajectory training in alignment quality under low computational budgets while substantially improving training efficiency. Flash-GRPO addresses two critical challenges: iso-temporal grouping eliminates timestep-confounded variance by enforcing prompt-wise temporal consistency, decoupling policy performance from timestep difficulty; temporal gradient rectification neutralizes the time-dependent scaling factor that causes vastly inconsistent gradient magnitudes across timesteps. Experiments on 1.3B to 14B parameter models validate Flash-GRPO's effectiveness, demonstrating substantial training acceleration with consistent stability and state-of-the-art alignment quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_15980 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization He, Xiaoxuan Fu, Siming Xue, Zeyue Wang, Weijie He, Ruizhe Li, Yuming Yin, Dacheng Dong, Shuai Huang, Haoyang Wang, Hongfa Duan, Nan Zhuang, Bohan Computer Vision and Pattern Recognition Group Relative Policy Optimization has emerged as essential for aligning video diffusion models with human preferences, but faces a critical computational bottleneck: training a 14B parametered model typically demands hundreds of GPU days per experiment. Existing efficiency methods reduce costs through sliding window subsampling training timesteps, but fundamentally compromise optimization, exhibiting severe instability and failing to reach full trajectory performance. We present Flash-GRPO, a single-step training framework that outperforms full trajectory training in alignment quality under low computational budgets while substantially improving training efficiency. Flash-GRPO addresses two critical challenges: iso-temporal grouping eliminates timestep-confounded variance by enforcing prompt-wise temporal consistency, decoupling policy performance from timestep difficulty; temporal gradient rectification neutralizes the time-dependent scaling factor that causes vastly inconsistent gradient magnitudes across timesteps. Experiments on 1.3B to 14B parameter models validate Flash-GRPO's effectiveness, demonstrating substantial training acceleration with consistent stability and state-of-the-art alignment quality. |
| title | Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.15980 |