Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.13395 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917757557145600 |
|---|---|
| author | Xu, Yangyang Shao, Wenqi Du, Yong Zhu, Haiming Zhou, Yang Luo, Ping He, Shengfeng |
| author_facet | Xu, Yangyang Shao, Wenqi Du, Yong Zhu, Haiming Zhou, Yang Luo, Ping He, Shengfeng |
| contents | Recent advancements in text-guided diffusion models have unlocked powerful image manipulation capabilities, yet balancing reconstruction fidelity and editability for real images remains a significant challenge. In this work, we introduce \textbf{T}ask-\textbf{O}riented \textbf{D}iffusion \textbf{I}nversion (\textbf{TODInv}), a novel framework that inverts and edits real images tailored to specific editing tasks by optimizing prompt embeddings within the extended \(\mathcal{P}^*\) space. By leveraging distinct embeddings across different U-Net layers and time steps, TODInv seamlessly integrates inversion and editing through reciprocal optimization, ensuring both high fidelity and precise editability. This hierarchical editing mechanism categorizes tasks into structure, appearance, and global edits, optimizing only those embeddings unaffected by the current editing task. Extensive experiments on benchmark dataset reveal TODInv's superior performance over existing methods, delivering both quantitative and qualitative enhancements while showcasing its versatility with few-step diffusion model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_13395 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Task-Oriented Diffusion Inversion for High-Fidelity Text-based Editing Xu, Yangyang Shao, Wenqi Du, Yong Zhu, Haiming Zhou, Yang Luo, Ping He, Shengfeng Computer Vision and Pattern Recognition Recent advancements in text-guided diffusion models have unlocked powerful image manipulation capabilities, yet balancing reconstruction fidelity and editability for real images remains a significant challenge. In this work, we introduce \textbf{T}ask-\textbf{O}riented \textbf{D}iffusion \textbf{I}nversion (\textbf{TODInv}), a novel framework that inverts and edits real images tailored to specific editing tasks by optimizing prompt embeddings within the extended \(\mathcal{P}^*\) space. By leveraging distinct embeddings across different U-Net layers and time steps, TODInv seamlessly integrates inversion and editing through reciprocal optimization, ensuring both high fidelity and precise editability. This hierarchical editing mechanism categorizes tasks into structure, appearance, and global edits, optimizing only those embeddings unaffected by the current editing task. Extensive experiments on benchmark dataset reveal TODInv's superior performance over existing methods, delivering both quantitative and qualitative enhancements while showcasing its versatility with few-step diffusion model. |
| title | Task-Oriented Diffusion Inversion for High-Fidelity Text-based Editing |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.13395 |