TokenVerse: Versatile Multi-concept Personalization in Token Modulation Space
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917898587471872 |
|---|---|
| author | Garibi, Daniel Yadin, Shahar Paiss, Roni Tov, Omer Zada, Shiran Ephrat, Ariel Michaeli, Tomer Mosseri, Inbar Dekel, Tali |
| author_facet | Garibi, Daniel Yadin, Shahar Paiss, Roni Tov, Omer Zada, Shiran Ephrat, Ariel Michaeli, Tomer Mosseri, Inbar Dekel, Tali |
| contents | We present TokenVerse -- a method for multi-concept personalization, leveraging a pre-trained text-to-image diffusion model. Our framework can disentangle complex visual elements and attributes from as little as a single image, while enabling seamless plug-and-play generation of combinations of concepts extracted from multiple images. As opposed to existing works, TokenVerse can handle multiple images with multiple concepts each, and supports a wide-range of concepts, including objects, accessories, materials, pose, and lighting. Our work exploits a DiT-based text-to-image model, in which the input text affects the generation through both attention and modulation (shift and scale). We observe that the modulation space is semantic and enables localized control over complex concepts. Building on this insight, we devise an optimization-based framework that takes as input an image and a text description, and finds for each word a distinct direction in the modulation space. These directions can then be used to generate new images that combine the learned concepts in a desired configuration. We demonstrate the effectiveness of TokenVerse in challenging personalization settings, and showcase its advantages over existing methods. project's webpage in https://token-verse.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_12224 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TokenVerse: Versatile Multi-concept Personalization in Token Modulation Space Garibi, Daniel Yadin, Shahar Paiss, Roni Tov, Omer Zada, Shiran Ephrat, Ariel Michaeli, Tomer Mosseri, Inbar Dekel, Tali Computer Vision and Pattern Recognition We present TokenVerse -- a method for multi-concept personalization, leveraging a pre-trained text-to-image diffusion model. Our framework can disentangle complex visual elements and attributes from as little as a single image, while enabling seamless plug-and-play generation of combinations of concepts extracted from multiple images. As opposed to existing works, TokenVerse can handle multiple images with multiple concepts each, and supports a wide-range of concepts, including objects, accessories, materials, pose, and lighting. Our work exploits a DiT-based text-to-image model, in which the input text affects the generation through both attention and modulation (shift and scale). We observe that the modulation space is semantic and enables localized control over complex concepts. Building on this insight, we devise an optimization-based framework that takes as input an image and a text description, and finds for each word a distinct direction in the modulation space. These directions can then be used to generate new images that combine the learned concepts in a desired configuration. We demonstrate the effectiveness of TokenVerse in challenging personalization settings, and showcase its advantages over existing methods. project's webpage in https://token-verse.github.io/ |
| title | TokenVerse: Versatile Multi-concept Personalization in Token Modulation Space |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.12224 |