Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2604.18348 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908980395114496 |
|---|---|
| author | Tan, Haoyue Wang, Shengnan Qiao, Yulin Zhang, Juncheng Bai, Youhui Gong, Ping Jin, Zewen Li, Cheng |
| author_facet | Tan, Haoyue Wang, Shengnan Qiao, Yulin Zhang, Juncheng Bai, Youhui Gong, Ping Jin, Zewen Li, Cheng |
| contents | Video diffusion transformers (DiTs) suffer from prohibitive inference latency due to quadratic attention complexity. Existing sparse attention methods either overlook semantic similarity or fail to adapt to heterogeneous token distributions across layers, leading to model performance degradation. We propose AdaCluster, a training-free adaptive clustering framework that accelerates the generation of DiTs while preserving accuracy. AdaCluster applies an angle-similarity-preserving clustering method to query vectors for higher compression, and designs a euclidean-similarity-preserving clustering method for keys, covering cluster number assignment, threshold-wise adaptive clustering, and efficient critical cluster selection. Experiments on CogVideoX-2B, HunyuanVideo, and Wan-2.1 on one A40 GPU demonstrate up to 1.67-4.31x speedup with negligible quality degradation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_18348 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AdaCluster: Adaptive Query-Key Clustering for Sparse Attention in Video Generation Tan, Haoyue Wang, Shengnan Qiao, Yulin Zhang, Juncheng Bai, Youhui Gong, Ping Jin, Zewen Li, Cheng Computer Vision and Pattern Recognition Artificial Intelligence Video diffusion transformers (DiTs) suffer from prohibitive inference latency due to quadratic attention complexity. Existing sparse attention methods either overlook semantic similarity or fail to adapt to heterogeneous token distributions across layers, leading to model performance degradation. We propose AdaCluster, a training-free adaptive clustering framework that accelerates the generation of DiTs while preserving accuracy. AdaCluster applies an angle-similarity-preserving clustering method to query vectors for higher compression, and designs a euclidean-similarity-preserving clustering method for keys, covering cluster number assignment, threshold-wise adaptive clustering, and efficient critical cluster selection. Experiments on CogVideoX-2B, HunyuanVideo, and Wan-2.1 on one A40 GPU demonstrate up to 1.67-4.31x speedup with negligible quality degradation. |
| title | AdaCluster: Adaptive Query-Key Clustering for Sparse Attention in Video Generation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2604.18348 |