DreamBoothDPO: Improving Personalized Generation using Direct Preference Optimization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916761785335808 |
|---|---|
| author | Ayupov, Shamil Nakhodnov, Maksim Yaschenko, Anastasia Kuznetsov, Andrey Alanov, Aibek |
| author_facet | Ayupov, Shamil Nakhodnov, Maksim Yaschenko, Anastasia Kuznetsov, Andrey Alanov, Aibek |
| contents | Personalized diffusion models have shown remarkable success in Text-to-Image (T2I) generation by enabling the injection of user-defined concepts into diverse contexts. However, balancing concept fidelity with contextual alignment remains a challenging open problem. In this work, we propose an RL-based approach that leverages the diverse outputs of T2I models to address this issue. Our method eliminates the need for human-annotated scores by generating a synthetic paired dataset for DPO-like training using external quality metrics. These better-worse pairs are specifically constructed to improve both concept fidelity and prompt adherence. Moreover, our approach supports flexible adjustment of the trade-off between image fidelity and textual alignment. Through multi-step training, our approach outperforms a naive baseline in convergence speed and output quality. We conduct extensive qualitative and quantitative analysis, demonstrating the effectiveness of our method across various architectures and fine-tuning techniques. The source code can be found at https://github.com/ControlGenAI/DreamBoothDPO. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_20975 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DreamBoothDPO: Improving Personalized Generation using Direct Preference Optimization Ayupov, Shamil Nakhodnov, Maksim Yaschenko, Anastasia Kuznetsov, Andrey Alanov, Aibek Computer Vision and Pattern Recognition Personalized diffusion models have shown remarkable success in Text-to-Image (T2I) generation by enabling the injection of user-defined concepts into diverse contexts. However, balancing concept fidelity with contextual alignment remains a challenging open problem. In this work, we propose an RL-based approach that leverages the diverse outputs of T2I models to address this issue. Our method eliminates the need for human-annotated scores by generating a synthetic paired dataset for DPO-like training using external quality metrics. These better-worse pairs are specifically constructed to improve both concept fidelity and prompt adherence. Moreover, our approach supports flexible adjustment of the trade-off between image fidelity and textual alignment. Through multi-step training, our approach outperforms a naive baseline in convergence speed and output quality. We conduct extensive qualitative and quantitative analysis, demonstrating the effectiveness of our method across various architectures and fine-tuning techniques. The source code can be found at https://github.com/ControlGenAI/DreamBoothDPO. |
| title | DreamBoothDPO: Improving Personalized Generation using Direct Preference Optimization |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.20975 |