AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929377352089600 |
|---|---|
| author | Pang, Lianyu Yin, Jian Zhao, Baoquan Wu, Feize Wang, Fu Lee Li, Qing Mao, Xudong |
| author_facet | Pang, Lianyu Yin, Jian Zhao, Baoquan Wu, Feize Wang, Fu Lee Li, Qing Mao, Xudong |
| contents | Recent advances in text-to-image models have enabled high-quality personalized image synthesis of user-provided concepts with flexible textual control. In this work, we analyze the limitations of two primary techniques in text-to-image personalization: Textual Inversion and DreamBooth. When integrating the learned concept into new prompts, Textual Inversion tends to overfit the concept, while DreamBooth often overlooks it. We attribute these issues to the incorrect learning of the embedding alignment for the concept. We introduce AttnDreamBooth, a novel approach that addresses these issues by separately learning the embedding alignment, the attention map, and the subject identity in different training stages. We also introduce a cross-attention map regularization term to enhance the learning of the attention map. Our method demonstrates significant improvements in identity preservation and text alignment compared to the baseline methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_05000 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation Pang, Lianyu Yin, Jian Zhao, Baoquan Wu, Feize Wang, Fu Lee Li, Qing Mao, Xudong Computer Vision and Pattern Recognition Recent advances in text-to-image models have enabled high-quality personalized image synthesis of user-provided concepts with flexible textual control. In this work, we analyze the limitations of two primary techniques in text-to-image personalization: Textual Inversion and DreamBooth. When integrating the learned concept into new prompts, Textual Inversion tends to overfit the concept, while DreamBooth often overlooks it. We attribute these issues to the incorrect learning of the embedding alignment for the concept. We introduce AttnDreamBooth, a novel approach that addresses these issues by separately learning the embedding alignment, the attention map, and the subject identity in different training stages. We also introduce a cross-attention map regularization term to enhance the learning of the attention map. Our method demonstrates significant improvements in identity preservation and text alignment compared to the baseline methods. |
| title | AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.05000 |