Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.06345 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929207565615104 |
|---|---|
| author | Yu, Chang Peng, Junran Zhu, Xiangyu Zhang, Zhaoxiang Tian, Qi Lei, Zhen |
| author_facet | Yu, Chang Peng, Junran Zhu, Xiangyu Zhang, Zhaoxiang Tian, Qi Lei, Zhen |
| contents | The text-to-image synthesis by diffusion models has recently shown remarkable performance in generating high-quality images. Although performs well for simple texts, the models may get confused when faced with complex texts that contain multiple objects or spatial relationships. To get the desired images, a feasible way is to manually adjust the textual descriptions, i.e., narrating the texts or adding some words, which is labor-consuming. In this paper, we propose a framework to learn the proper textual descriptions for diffusion models through prompt learning. By utilizing the quality guidance and the semantic guidance derived from the pre-trained diffusion model, our method can effectively learn the prompts to improve the matches between the input text and the generated images. Extensive experiments and analyses have validated the effectiveness of the proposed method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_06345 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Seek for Incantations: Towards Accurate Text-to-Image Diffusion Synthesis through Prompt Engineering Yu, Chang Peng, Junran Zhu, Xiangyu Zhang, Zhaoxiang Tian, Qi Lei, Zhen Computer Vision and Pattern Recognition The text-to-image synthesis by diffusion models has recently shown remarkable performance in generating high-quality images. Although performs well for simple texts, the models may get confused when faced with complex texts that contain multiple objects or spatial relationships. To get the desired images, a feasible way is to manually adjust the textual descriptions, i.e., narrating the texts or adding some words, which is labor-consuming. In this paper, we propose a framework to learn the proper textual descriptions for diffusion models through prompt learning. By utilizing the quality guidance and the semantic guidance derived from the pre-trained diffusion model, our method can effectively learn the prompts to improve the matches between the input text and the generated images. Extensive experiments and analyses have validated the effectiveness of the proposed method. |
| title | Seek for Incantations: Towards Accurate Text-to-Image Diffusion Synthesis through Prompt Engineering |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.06345 |