Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915768177786880 |
|---|---|
| author | Luo, Yifu Hu, Xinhao Fan, Keyu Sun, Haoyuan Chen, Zeyu Xia, Bo Zhang, Tiantian Chang, Yongzhe Wang, Xueqian |
| author_facet | Luo, Yifu Hu, Xinhao Fan, Keyu Sun, Haoyuan Chen, Zeyu Xia, Bo Zhang, Tiantian Chang, Yongzhe Wang, Xueqian |
| contents | Reinforcement learning (RL) has garnered increasing attention in text-to-image (T2I) generation. However, most existing RL approaches are tailored to either diffusion models or autoregressive models, overlooking an important alternative: masked generative models. In this work, we propose Mask-GRPO, the first method to incorporate Group Relative Policy Optimization (GRPO)-based RL into this overlooked paradigm. Our core insight is to redefine the transition probability, which is different from current approaches, and formulate the unmasking process as a multi-step decision-making problem. To further enhance our method, we explore several useful strategies, including removing the KL constraint, applying the reduction strategy, and filtering out low-quality samples. Using Mask-GRPO, we improve a base model, Show-o, with substantial improvements on standard T2I benchmarks and preference alignment, outperforming existing state-of-the-art approaches. The code is available on https://github.com/xingzhejun/Mask-GRPO |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_13418 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation Luo, Yifu Hu, Xinhao Fan, Keyu Sun, Haoyuan Chen, Zeyu Xia, Bo Zhang, Tiantian Chang, Yongzhe Wang, Xueqian Computer Vision and Pattern Recognition Reinforcement learning (RL) has garnered increasing attention in text-to-image (T2I) generation. However, most existing RL approaches are tailored to either diffusion models or autoregressive models, overlooking an important alternative: masked generative models. In this work, we propose Mask-GRPO, the first method to incorporate Group Relative Policy Optimization (GRPO)-based RL into this overlooked paradigm. Our core insight is to redefine the transition probability, which is different from current approaches, and formulate the unmasking process as a multi-step decision-making problem. To further enhance our method, we explore several useful strategies, including removing the KL constraint, applying the reduction strategy, and filtering out low-quality samples. Using Mask-GRPO, we improve a base model, Show-o, with substantial improvements on standard T2I benchmarks and preference alignment, outperforming existing state-of-the-art approaches. The code is available on https://github.com/xingzhejun/Mask-GRPO |
| title | Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.13418 |