PALP: Prompt Aligned Personalization of Text-to-Image Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911755358175232 |
|---|---|
| author | Arar, Moab Voynov, Andrey Hertz, Amir Avrahami, Omri Fruchter, Shlomi Pritch, Yael Cohen-Or, Daniel Shamir, Ariel |
| author_facet | Arar, Moab Voynov, Andrey Hertz, Amir Avrahami, Omri Fruchter, Shlomi Pritch, Yael Cohen-Or, Daniel Shamir, Ariel |
| contents | Content creators often aim to create personalized images using personal subjects that go beyond the capabilities of conventional text-to-image models. Additionally, they may want the resulting image to encompass a specific location, style, ambiance, and more. Existing personalization methods may compromise personalization ability or the alignment to complex textual prompts. This trade-off can impede the fulfillment of user prompts and subject fidelity. We propose a new approach focusing on personalization methods for a \emph{single} prompt to address this issue. We term our approach prompt-aligned personalization. While this may seem restrictive, our method excels in improving text alignment, enabling the creation of images with complex and intricate prompts, which may pose a challenge for current techniques. In particular, our method keeps the personalized model aligned with a target prompt using an additional score distillation sampling term. We demonstrate the versatility of our method in multi- and single-shot settings and further show that it can compose multiple subjects or use inspiration from reference images, such as artworks. We compare our approach quantitatively and qualitatively with existing baselines and state-of-the-art techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_06105 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PALP: Prompt Aligned Personalization of Text-to-Image Models Arar, Moab Voynov, Andrey Hertz, Amir Avrahami, Omri Fruchter, Shlomi Pritch, Yael Cohen-Or, Daniel Shamir, Ariel Computer Vision and Pattern Recognition Computation and Language Graphics Machine Learning Content creators often aim to create personalized images using personal subjects that go beyond the capabilities of conventional text-to-image models. Additionally, they may want the resulting image to encompass a specific location, style, ambiance, and more. Existing personalization methods may compromise personalization ability or the alignment to complex textual prompts. This trade-off can impede the fulfillment of user prompts and subject fidelity. We propose a new approach focusing on personalization methods for a \emph{single} prompt to address this issue. We term our approach prompt-aligned personalization. While this may seem restrictive, our method excels in improving text alignment, enabling the creation of images with complex and intricate prompts, which may pose a challenge for current techniques. In particular, our method keeps the personalized model aligned with a target prompt using an additional score distillation sampling term. We demonstrate the versatility of our method in multi- and single-shot settings and further show that it can compose multiple subjects or use inspiration from reference images, such as artworks. We compare our approach quantitatively and qualitatively with existing baselines and state-of-the-art techniques. |
| title | PALP: Prompt Aligned Personalization of Text-to-Image Models |
| topic | Computer Vision and Pattern Recognition Computation and Language Graphics Machine Learning |
| url | https://arxiv.org/abs/2401.06105 |