Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2512.22234 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914234744438784 |
|---|---|
| author | Zhu, Ying Wan, Jiaxin Liu, Xiaoran He, Siyang Wang, Qiqi Guo, Xu Liang, Tianyi Huang, Zengfeng He, Ziwei Qiu, Xipeng |
| author_facet | Zhu, Ying Wan, Jiaxin Liu, Xiaoran He, Siyang Wang, Qiqi Guo, Xu Liang, Tianyi Huang, Zengfeng He, Ziwei Qiu, Xipeng |
| contents | Diffusion Language Models (dLLMs) have emerged as promising alternatives to Auto-Regressive (AR) models. While recent efforts have validated their pre-training potential and accelerated inference speeds, the post-training landscape for dLLMs remains underdeveloped. Existing methods suffer from computational inefficiency and objective mismatches between training and inference, severely limiting performance on complex reasoning tasks such as mathematics. To address this, we introduce DiRL, an efficient post-training framework that tightly integrates FlexAttention-accelerated blockwise training with LMDeploy-optimized inference. This architecture enables a streamlined online model update loop, facilitating efficient two-stage post-training (Supervised Fine-Tuning followed by Reinforcement Learning). Building on this framework, we propose DiPO, the first unbiased Group Relative Policy Optimization (GRPO) implementation tailored for dLLMs. We validate our approach by training DiRL-8B-Instruct on high-quality math data. Our model achieves state-of-the-art math performance among dLLMs and surpasses comparable models in the Qwen2.5 series on several benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_22234 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DiRL: An Efficient Post-Training Framework for Diffusion Language Models Zhu, Ying Wan, Jiaxin Liu, Xiaoran He, Siyang Wang, Qiqi Guo, Xu Liang, Tianyi Huang, Zengfeng He, Ziwei Qiu, Xipeng Machine Learning Artificial Intelligence Diffusion Language Models (dLLMs) have emerged as promising alternatives to Auto-Regressive (AR) models. While recent efforts have validated their pre-training potential and accelerated inference speeds, the post-training landscape for dLLMs remains underdeveloped. Existing methods suffer from computational inefficiency and objective mismatches between training and inference, severely limiting performance on complex reasoning tasks such as mathematics. To address this, we introduce DiRL, an efficient post-training framework that tightly integrates FlexAttention-accelerated blockwise training with LMDeploy-optimized inference. This architecture enables a streamlined online model update loop, facilitating efficient two-stage post-training (Supervised Fine-Tuning followed by Reinforcement Learning). Building on this framework, we propose DiPO, the first unbiased Group Relative Policy Optimization (GRPO) implementation tailored for dLLMs. We validate our approach by training DiRL-8B-Instruct on high-quality math data. Our model achieves state-of-the-art math performance among dLLMs and surpasses comparable models in the Qwen2.5 series on several benchmarks. |
| title | DiRL: An Efficient Post-Training Framework for Diffusion Language Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2512.22234 |