Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.01356 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917655779213312 |
|---|---|
| author | Chan, Kelvin C. K. Zhao, Yang Jia, Xuhui Yang, Ming-Hsuan Wang, Huisheng |
| author_facet | Chan, Kelvin C. K. Zhao, Yang Jia, Xuhui Yang, Ming-Hsuan Wang, Huisheng |
| contents | In subject-driven text-to-image synthesis, the synthesis process tends to be heavily influenced by the reference images provided by users, often overlooking crucial attributes detailed in the text prompt. In this work, we propose Subject-Agnostic Guidance (SAG), a simple yet effective solution to remedy the problem. We show that through constructing a subject-agnostic condition and applying our proposed dual classifier-free guidance, one could obtain outputs consistent with both the given subject and input text prompts. We validate the efficacy of our approach through both optimization-based and encoder-based methods. Additionally, we demonstrate its applicability in second-order customization methods, where an encoder-based model is fine-tuned with DreamBooth. Our approach is conceptually simple and requires only minimal code modifications, but leads to substantial quality improvements, as evidenced by our evaluations and user studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_01356 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improving Subject-Driven Image Synthesis with Subject-Agnostic Guidance Chan, Kelvin C. K. Zhao, Yang Jia, Xuhui Yang, Ming-Hsuan Wang, Huisheng Computer Vision and Pattern Recognition In subject-driven text-to-image synthesis, the synthesis process tends to be heavily influenced by the reference images provided by users, often overlooking crucial attributes detailed in the text prompt. In this work, we propose Subject-Agnostic Guidance (SAG), a simple yet effective solution to remedy the problem. We show that through constructing a subject-agnostic condition and applying our proposed dual classifier-free guidance, one could obtain outputs consistent with both the given subject and input text prompts. We validate the efficacy of our approach through both optimization-based and encoder-based methods. Additionally, we demonstrate its applicability in second-order customization methods, where an encoder-based model is fine-tuned with DreamBooth. Our approach is conceptually simple and requires only minimal code modifications, but leads to substantial quality improvements, as evidenced by our evaluations and user studies. |
| title | Improving Subject-Driven Image Synthesis with Subject-Agnostic Guidance |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.01356 |