RectifID: Personalizing Rectified Flow with Anchored Classifier Guidance
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910736776691712 |
|---|---|
| author | Sun, Zhicheng Yang, Zhenhao Jin, Yang Chi, Haozhe Xu, Kun Xu, Kun Chen, Liwei Jiang, Hao Song, Yang Gai, Kun Mu, Yadong |
| author_facet | Sun, Zhicheng Yang, Zhenhao Jin, Yang Chi, Haozhe Xu, Kun Xu, Kun Chen, Liwei Jiang, Hao Song, Yang Gai, Kun Mu, Yadong |
| contents | Customizing diffusion models to generate identity-preserving images from user-provided reference images is an intriguing new problem. The prevalent approaches typically require training on extensive domain-specific images to achieve identity preservation, which lacks flexibility across different use cases. To address this issue, we exploit classifier guidance, a training-free technique that steers diffusion models using an existing classifier, for personalized image generation. Our study shows that based on a recent rectified flow framework, the major limitation of vanilla classifier guidance in requiring a special classifier can be resolved with a simple fixed-point solution, allowing flexible personalization with off-the-shelf image discriminators. Moreover, its solving procedure proves to be stable when anchored to a reference flow trajectory, with a convergence guarantee. The derived method is implemented on rectified flow with different off-the-shelf image discriminators, delivering advantageous personalization results for human faces, live subjects, and certain objects. Code is available at https://github.com/feifeiobama/RectifID. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_14677 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RectifID: Personalizing Rectified Flow with Anchored Classifier Guidance Sun, Zhicheng Yang, Zhenhao Jin, Yang Chi, Haozhe Xu, Kun Xu, Kun Chen, Liwei Jiang, Hao Song, Yang Gai, Kun Mu, Yadong Computer Vision and Pattern Recognition Machine Learning Customizing diffusion models to generate identity-preserving images from user-provided reference images is an intriguing new problem. The prevalent approaches typically require training on extensive domain-specific images to achieve identity preservation, which lacks flexibility across different use cases. To address this issue, we exploit classifier guidance, a training-free technique that steers diffusion models using an existing classifier, for personalized image generation. Our study shows that based on a recent rectified flow framework, the major limitation of vanilla classifier guidance in requiring a special classifier can be resolved with a simple fixed-point solution, allowing flexible personalization with off-the-shelf image discriminators. Moreover, its solving procedure proves to be stable when anchored to a reference flow trajectory, with a convergence guarantee. The derived method is implemented on rectified flow with different off-the-shelf image discriminators, delivering advantageous personalization results for human faces, live subjects, and certain objects. Code is available at https://github.com/feifeiobama/RectifID. |
| title | RectifID: Personalizing Rectified Flow with Anchored Classifier Guidance |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2405.14677 |