Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.18263 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915948038979584 |
|---|---|
| author | Huang, Ziwei Shu, Ying Fang, Hao Long, Quanyu Wang, Wenya Guo, Qiushi Ge, Tiezheng Gan, Leilei |
| author_facet | Huang, Ziwei Shu, Ying Fang, Hao Long, Quanyu Wang, Wenya Guo, Qiushi Ge, Tiezheng Gan, Leilei |
| contents | Subject-driven image generation models face a fundamental trade-off between identity preservation (fidelity) and prompt adherence (editability). While online reinforcement learning (RL), specifically GPRO, offers a promising solution, we find that a naive application of GRPO leads to competitive degradation, as the simple linear aggregation of rewards with static weights causes conflicting gradient signals and a misalignment with the temporal dynamics of the diffusion process. To overcome these limitations, we propose Customized-GRPO, a novel framework featuring two key innovations: (i) Synergy-Aware Reward Shaping (SARS), a non-linear mechanism that explicitly penalizes conflicted reward signals and amplifies synergistic ones, providing a sharper and more decisive gradient. (ii) Time-Aware Dynamic Weighting (TDW), which aligns the optimization pressure with the model's temporal dynamics by prioritizing prompt-following in the early, identity preservation in the later. Extensive experiments demonstrate that our method significantly outperforms naive GRPO baselines, successfully mitigating competitive degradation. Our model achieves a superior balance, generating images that both preserve key identity features and accurately adhere to complex textual prompts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_18263 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Competition to Synergy: Unlocking Reinforcement Learning for Subject-Driven Image Generation Huang, Ziwei Shu, Ying Fang, Hao Long, Quanyu Wang, Wenya Guo, Qiushi Ge, Tiezheng Gan, Leilei Machine Learning Computer Vision and Pattern Recognition Graphics Subject-driven image generation models face a fundamental trade-off between identity preservation (fidelity) and prompt adherence (editability). While online reinforcement learning (RL), specifically GPRO, offers a promising solution, we find that a naive application of GRPO leads to competitive degradation, as the simple linear aggregation of rewards with static weights causes conflicting gradient signals and a misalignment with the temporal dynamics of the diffusion process. To overcome these limitations, we propose Customized-GRPO, a novel framework featuring two key innovations: (i) Synergy-Aware Reward Shaping (SARS), a non-linear mechanism that explicitly penalizes conflicted reward signals and amplifies synergistic ones, providing a sharper and more decisive gradient. (ii) Time-Aware Dynamic Weighting (TDW), which aligns the optimization pressure with the model's temporal dynamics by prioritizing prompt-following in the early, identity preservation in the later. Extensive experiments demonstrate that our method significantly outperforms naive GRPO baselines, successfully mitigating competitive degradation. Our model achieves a superior balance, generating images that both preserve key identity features and accurately adhere to complex textual prompts. |
| title | From Competition to Synergy: Unlocking Reinforcement Learning for Subject-Driven Image Generation |
| topic | Machine Learning Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2510.18263 |