DiTFastAttnV2: Head-wise Attention Compression for Multi-Modality Diffusion Transformers
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917971403735040 |
|---|---|
| author | Zhang, Hanling Su, Rundong Yuan, Zhihang Chen, Pengtao Fan, Mingzhu Shen Yibo Yan, Shengen Dai, Guohao Wang, Yu |
| author_facet | Zhang, Hanling Su, Rundong Yuan, Zhihang Chen, Pengtao Fan, Mingzhu Shen Yibo Yan, Shengen Dai, Guohao Wang, Yu |
| contents | Text-to-image generation models, especially Multimodal Diffusion Transformers (MMDiT), have shown remarkable progress in generating high-quality images. However, these models often face significant computational bottlenecks, particularly in attention mechanisms, which hinder their scalability and efficiency. In this paper, we introduce DiTFastAttnV2, a post-training compression method designed to accelerate attention in MMDiT. Through an in-depth analysis of MMDiT's attention patterns, we identify key differences from prior DiT-based methods and propose head-wise arrow attention and caching mechanisms to dynamically adjust attention heads, effectively bridging this gap. We also design an Efficient Fused Kernel for further acceleration. By leveraging local metric methods and optimization techniques, our approach significantly reduces the search time for optimal compression schemes to just minutes while maintaining generation quality. Furthermore, with the customized kernel, DiTFastAttnV2 achieves a 68% reduction in attention FLOPs and 1.5x end-to-end speedup on 2K image generation without compromising visual fidelity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_22796 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DiTFastAttnV2: Head-wise Attention Compression for Multi-Modality Diffusion Transformers Zhang, Hanling Su, Rundong Yuan, Zhihang Chen, Pengtao Fan, Mingzhu Shen Yibo Yan, Shengen Dai, Guohao Wang, Yu Computer Vision and Pattern Recognition Artificial Intelligence Text-to-image generation models, especially Multimodal Diffusion Transformers (MMDiT), have shown remarkable progress in generating high-quality images. However, these models often face significant computational bottlenecks, particularly in attention mechanisms, which hinder their scalability and efficiency. In this paper, we introduce DiTFastAttnV2, a post-training compression method designed to accelerate attention in MMDiT. Through an in-depth analysis of MMDiT's attention patterns, we identify key differences from prior DiT-based methods and propose head-wise arrow attention and caching mechanisms to dynamically adjust attention heads, effectively bridging this gap. We also design an Efficient Fused Kernel for further acceleration. By leveraging local metric methods and optimization techniques, our approach significantly reduces the search time for optimal compression schemes to just minutes while maintaining generation quality. Furthermore, with the customized kernel, DiTFastAttnV2 achieves a 68% reduction in attention FLOPs and 1.5x end-to-end speedup on 2K image generation without compromising visual fidelity. |
| title | DiTFastAttnV2: Head-wise Attention Compression for Multi-Modality Diffusion Transformers |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2503.22796 |