Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.02067 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910672094232576 |
|---|---|
| author | He, Jing Li, Haodong Hu, Yongzhe Shen, Guibao Cai, Yingjie Qiu, Weichao Chen, Ying-Cong |
| author_facet | He, Jing Li, Haodong Hu, Yongzhe Shen, Guibao Cai, Yingjie Qiu, Weichao Chen, Ying-Cong |
| contents | In the realm of image generation, creating customized images from visual prompt with additional textual instruction emerges as a promising endeavor. However, existing methods, both tuning-based and tuning-free, struggle with interpreting the subject-essential attributes from the visual prompt. This leads to subject-irrelevant attributes infiltrating the generation process, ultimately compromising the personalization quality in both editability and ID preservation. In this paper, we present DisEnvisioner, a novel approach for effectively extracting and enriching the subject-essential features while filtering out -irrelevant information, enabling exceptional customization performance, in a tuning-free manner and using only a single image. Specifically, the feature of the subject and other irrelevant components are effectively separated into distinctive visual tokens, enabling a much more accurate customization. Aiming to further improving the ID consistency, we enrich the disentangled features, sculpting them into more granular representations. Experiments demonstrate the superiority of our approach over existing methods in instruction response (editability), ID consistency, inference speed, and the overall image quality, highlighting the effectiveness and efficiency of DisEnvisioner. Project page: https://disenvisioner.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_02067 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DisEnvisioner: Disentangled and Enriched Visual Prompt for Customized Image Generation He, Jing Li, Haodong Hu, Yongzhe Shen, Guibao Cai, Yingjie Qiu, Weichao Chen, Ying-Cong Computer Vision and Pattern Recognition In the realm of image generation, creating customized images from visual prompt with additional textual instruction emerges as a promising endeavor. However, existing methods, both tuning-based and tuning-free, struggle with interpreting the subject-essential attributes from the visual prompt. This leads to subject-irrelevant attributes infiltrating the generation process, ultimately compromising the personalization quality in both editability and ID preservation. In this paper, we present DisEnvisioner, a novel approach for effectively extracting and enriching the subject-essential features while filtering out -irrelevant information, enabling exceptional customization performance, in a tuning-free manner and using only a single image. Specifically, the feature of the subject and other irrelevant components are effectively separated into distinctive visual tokens, enabling a much more accurate customization. Aiming to further improving the ID consistency, we enrich the disentangled features, sculpting them into more granular representations. Experiments demonstrate the superiority of our approach over existing methods in instruction response (editability), ID consistency, inference speed, and the overall image quality, highlighting the effectiveness and efficiency of DisEnvisioner. Project page: https://disenvisioner.github.io/. |
| title | DisEnvisioner: Disentangled and Enriched Visual Prompt for Customized Image Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.02067 |