Fast Personalized Text-to-Image Syntheses With Attention Injection
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929281126367232 |
|---|---|
| author | Zhang, Yuxuan Song, Yiren Yu, Jinpeng Pan, Han Jing, Zhongliang |
| author_facet | Zhang, Yuxuan Song, Yiren Yu, Jinpeng Pan, Han Jing, Zhongliang |
| contents | Currently, personalized image generation methods mostly require considerable time to finetune and often overfit the concept resulting in generated images that are similar to custom concepts but difficult to edit by prompts. We propose an effective and fast approach that could balance the text-image consistency and identity consistency of the generated image and reference image. Our method can generate personalized images without any fine-tuning while maintaining the inherent text-to-image generation ability of diffusion models. Given a prompt and a reference image, we merge the custom concept into generated images by manipulating cross-attention and self-attention layers of the original diffusion model to generate personalized images that match the text description. Comprehensive experiments highlight the superiority of our method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_11284 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Fast Personalized Text-to-Image Syntheses With Attention Injection Zhang, Yuxuan Song, Yiren Yu, Jinpeng Pan, Han Jing, Zhongliang Computer Vision and Pattern Recognition Currently, personalized image generation methods mostly require considerable time to finetune and often overfit the concept resulting in generated images that are similar to custom concepts but difficult to edit by prompts. We propose an effective and fast approach that could balance the text-image consistency and identity consistency of the generated image and reference image. Our method can generate personalized images without any fine-tuning while maintaining the inherent text-to-image generation ability of diffusion models. Given a prompt and a reference image, we merge the custom concept into generated images by manipulating cross-attention and self-attention layers of the original diffusion model to generate personalized images that match the text description. Comprehensive experiments highlight the superiority of our method. |
| title | Fast Personalized Text-to-Image Syntheses With Attention Injection |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.11284 |