Rethinking Global Text Conditioning in Diffusion Transformers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918330415185920 |
|---|---|
| author | Starodubcev, Nikita Pakhomov, Daniil Wu, Zongze Drobyshevskiy, Ilya Liu, Yuchen Wang, Zhonghao Zhou, Yuqian Lin, Zhe Baranchuk, Dmitry |
| author_facet | Starodubcev, Nikita Pakhomov, Daniil Wu, Zongze Drobyshevskiy, Ilya Liu, Yuchen Wang, Zhonghao Zhou, Yuqian Lin, Zhe Baranchuk, Dmitry |
| contents | Diffusion transformers typically incorporate textual information via attention layers and a modulation mechanism using a pooled text embedding. Nevertheless, recent approaches discard modulation-based text conditioning and rely exclusively on attention. In this paper, we address whether modulation-based text conditioning is necessary and whether it can provide any performance advantage. Our analysis shows that, in its conventional usage, the pooled embedding contributes little to overall performance, suggesting that attention alone is generally sufficient for faithfully propagating prompt information. However, we reveal that the pooled embedding can provide significant gains when used from a different perspective-serving as guidance and enabling controllable shifts toward more desirable properties. This approach is training-free, simple to implement, incurs negligible runtime overhead, and can be applied to various diffusion models, bringing improvements across diverse tasks, including text-to-image/video generation and image editing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_09268 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Rethinking Global Text Conditioning in Diffusion Transformers Starodubcev, Nikita Pakhomov, Daniil Wu, Zongze Drobyshevskiy, Ilya Liu, Yuchen Wang, Zhonghao Zhou, Yuqian Lin, Zhe Baranchuk, Dmitry Computer Vision and Pattern Recognition Diffusion transformers typically incorporate textual information via attention layers and a modulation mechanism using a pooled text embedding. Nevertheless, recent approaches discard modulation-based text conditioning and rely exclusively on attention. In this paper, we address whether modulation-based text conditioning is necessary and whether it can provide any performance advantage. Our analysis shows that, in its conventional usage, the pooled embedding contributes little to overall performance, suggesting that attention alone is generally sufficient for faithfully propagating prompt information. However, we reveal that the pooled embedding can provide significant gains when used from a different perspective-serving as guidance and enabling controllable shifts toward more desirable properties. This approach is training-free, simple to implement, incurs negligible runtime overhead, and can be applied to various diffusion models, bringing improvements across diverse tasks, including text-to-image/video generation and image editing. |
| title | Rethinking Global Text Conditioning in Diffusion Transformers |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2602.09268 |