Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.16501 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916261215076352 |
|---|---|
| author | Zhong, Linhao Hong, Yan Chen, Wentao Zhou, Binglin Zhang, Yiyi Zhang, Jianfu Zhang, Liqing |
| author_facet | Zhong, Linhao Hong, Yan Chen, Wentao Zhou, Binglin Zhang, Yiyi Zhang, Jianfu Zhang, Liqing |
| contents | Text-to-image generation models have seen considerable advancement, catering to the increasing interest in personalized image creation. Current customization techniques often necessitate users to provide multiple images (typically 3-5) for each customized object, along with the classification of these objects and descriptive textual prompts for scenes. This paper questions whether the process can be made more user-friendly and the customization more intricate. We propose a method where users need only provide images along with text for each customization topic, and necessitates only a single image per visual concept. We introduce the concept of a ``multi-modal prompt'', a novel integration of text and images tailored to each customization concept, which simplifies user interaction and facilitates precise customization of both objects and scenes. Our proposed paradigm for customized text-to-image generation surpasses existing finetune-based methods in user-friendliness and the ability to customize complex objects with user-friendly inputs. Our code is available at $\href{https://github.com/zhongzero/Multi-Modal-Prompt}{https://github.com/zhongzero/Multi-Modal-Prompt}$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_16501 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | User-Friendly Customized Generation with Multi-Modal Prompts Zhong, Linhao Hong, Yan Chen, Wentao Zhou, Binglin Zhang, Yiyi Zhang, Jianfu Zhang, Liqing Computer Vision and Pattern Recognition Text-to-image generation models have seen considerable advancement, catering to the increasing interest in personalized image creation. Current customization techniques often necessitate users to provide multiple images (typically 3-5) for each customized object, along with the classification of these objects and descriptive textual prompts for scenes. This paper questions whether the process can be made more user-friendly and the customization more intricate. We propose a method where users need only provide images along with text for each customization topic, and necessitates only a single image per visual concept. We introduce the concept of a ``multi-modal prompt'', a novel integration of text and images tailored to each customization concept, which simplifies user interaction and facilitates precise customization of both objects and scenes. Our proposed paradigm for customized text-to-image generation surpasses existing finetune-based methods in user-friendliness and the ability to customize complex objects with user-friendly inputs. Our code is available at $\href{https://github.com/zhongzero/Multi-Modal-Prompt}{https://github.com/zhongzero/Multi-Modal-Prompt}$. |
| title | User-Friendly Customized Generation with Multi-Modal Prompts |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.16501 |