Towards Transformer-Based Aligned Generation with Self-Coherence Guidance
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910888806580224 |
|---|---|
| author | Wang, Shulei Lin, Wang Huang, Hai Wang, Hanting Cai, Sihang Han, WenKang Jin, Tao Chen, Jingyuan Sun, Jiacheng Zhu, Jieming Zhao, Zhou |
| author_facet | Wang, Shulei Lin, Wang Huang, Hai Wang, Hanting Cai, Sihang Han, WenKang Jin, Tao Chen, Jingyuan Sun, Jiacheng Zhu, Jieming Zhao, Zhou |
| contents | We introduce a novel, training-free approach for enhancing alignment in Transformer-based Text-Guided Diffusion Models (TGDMs). Existing TGDMs often struggle to generate semantically aligned images, particularly when dealing with complex text prompts or multi-concept attribute binding challenges. Previous U-Net-based methods primarily optimized the latent space, but their direct application to Transformer-based architectures has shown limited effectiveness. Our method addresses these challenges by directly optimizing cross-attention maps during the generation process. Specifically, we introduce Self-Coherence Guidance, a method that dynamically refines attention maps using masks derived from previous denoising steps, ensuring precise alignment without additional training. To validate our approach, we constructed more challenging benchmarks for evaluating coarse-grained attribute binding, fine-grained attribute binding, and style binding. Experimental results demonstrate the superior performance of our method, significantly surpassing other state-of-the-art methods across all evaluated tasks. Our code is available at https://scg-diffusion.github.io/scg-diffusion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_17675 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Transformer-Based Aligned Generation with Self-Coherence Guidance Wang, Shulei Lin, Wang Huang, Hai Wang, Hanting Cai, Sihang Han, WenKang Jin, Tao Chen, Jingyuan Sun, Jiacheng Zhu, Jieming Zhao, Zhou Computer Vision and Pattern Recognition We introduce a novel, training-free approach for enhancing alignment in Transformer-based Text-Guided Diffusion Models (TGDMs). Existing TGDMs often struggle to generate semantically aligned images, particularly when dealing with complex text prompts or multi-concept attribute binding challenges. Previous U-Net-based methods primarily optimized the latent space, but their direct application to Transformer-based architectures has shown limited effectiveness. Our method addresses these challenges by directly optimizing cross-attention maps during the generation process. Specifically, we introduce Self-Coherence Guidance, a method that dynamically refines attention maps using masks derived from previous denoising steps, ensuring precise alignment without additional training. To validate our approach, we constructed more challenging benchmarks for evaluating coarse-grained attribute binding, fine-grained attribute binding, and style binding. Experimental results demonstrate the superior performance of our method, significantly surpassing other state-of-the-art methods across all evaluated tasks. Our code is available at https://scg-diffusion.github.io/scg-diffusion. |
| title | Towards Transformer-Based Aligned Generation with Self-Coherence Guidance |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.17675 |