Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.03738 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913777692180480 |
|---|---|
| author | Hua, Litao Liu, Fan Su, Jie Miao, Xingyu Ouyang, Zizhou Wang, Zeyu Hu, Runze Wen, Zhenyu Zhai, Bing Long, Yang Duan, Haoran Zhou, Yuan |
| author_facet | Hua, Litao Liu, Fan Su, Jie Miao, Xingyu Ouyang, Zizhou Wang, Zeyu Hu, Runze Wen, Zhenyu Zhai, Bing Long, Yang Duan, Haoran Zhou, Yuan |
| contents | Attention mechanisms have become a foundational component in diffusion models, significantly influencing their capacity across a wide range of generative and discriminative tasks. This paper presents a comprehensive survey of attention within diffusion models, systematically analysing its roles, design patterns, and operations across different modalities and tasks. We propose a unified taxonomy that categorises attention-related modifications into parts according to the structural components they affect, offering a clear lens through which to understand their functional diversity. In addition to reviewing architectural innovations, we examine how attention mechanisms contribute to performance improvements in diverse applications. We also identify current limitations and underexplored areas, and outline potential directions for future research. Our study provides valuable insights into the evolving landscape of diffusion models, with a particular focus on the integrative and ubiquitous role of attention. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_03738 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Attention in Diffusion Model: A Survey Hua, Litao Liu, Fan Su, Jie Miao, Xingyu Ouyang, Zizhou Wang, Zeyu Hu, Runze Wen, Zhenyu Zhai, Bing Long, Yang Duan, Haoran Zhou, Yuan Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Attention mechanisms have become a foundational component in diffusion models, significantly influencing their capacity across a wide range of generative and discriminative tasks. This paper presents a comprehensive survey of attention within diffusion models, systematically analysing its roles, design patterns, and operations across different modalities and tasks. We propose a unified taxonomy that categorises attention-related modifications into parts according to the structural components they affect, offering a clear lens through which to understand their functional diversity. In addition to reviewing architectural innovations, we examine how attention mechanisms contribute to performance improvements in diverse applications. We also identify current limitations and underexplored areas, and outline potential directions for future research. Our study provides valuable insights into the evolving landscape of diffusion models, with a particular focus on the integrative and ubiquitous role of attention. |
| title | Attention in Diffusion Model: A Survey |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.03738 |