Guardado en:
| Autores principales: | , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2506.04648 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912417085128704 |
|---|---|
| author | Liu, Akide Zhang, Zeyu Li, Zhexin Bai, Xuehai Han, Yizeng Tang, Jiasheng Xing, Yuanjie Wu, Jichao Yang, Mingyang Chen, Weihua He, Jiahao He, Yuanyu Wang, Fan Haffari, Gholamreza Zhuang, Bohan |
| author_facet | Liu, Akide Zhang, Zeyu Li, Zhexin Bai, Xuehai Han, Yizeng Tang, Jiasheng Xing, Yuanjie Wu, Jichao Yang, Mingyang Chen, Weihua He, Jiahao He, Yuanyu Wang, Fan Haffari, Gholamreza Zhuang, Bohan |
| contents | Diffusion generative models have become the standard for producing high-quality, coherent video content, yet their slow inference speeds and high computational demands hinder practical deployment. Although both quantization and sparsity can independently accelerate inference while maintaining generation quality, naively combining these techniques in existing training-free approaches leads to significant performance degradation due to the lack of joint optimization. We introduce FPSAttention, a novel training-aware co-design of FP8 quantization and sparsity for video generation, with a focus on the 3D bi-directional attention mechanism. Our approach features three key innovations: 1) A unified 3D tile-wise granularity that simultaneously supports both quantization and sparsity; 2) A denoising step-aware strategy that adapts to the noise schedule, addressing the strong correlation between quantization/sparsity errors and denoising steps; 3) A native, hardware-friendly kernel that leverages FlashAttention and is implemented with optimized Hopper architecture features for highly efficient execution. Trained on Wan2.1's 1.3B and 14B models and evaluated on the VBench benchmark, FPSAttention achieves a 7.09x kernel speedup for attention operations and a 4.96x end-to-end speedup for video generation compared to the BF16 baseline at 720p resolution-without sacrificing generation quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_04648 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusion Liu, Akide Zhang, Zeyu Li, Zhexin Bai, Xuehai Han, Yizeng Tang, Jiasheng Xing, Yuanjie Wu, Jichao Yang, Mingyang Chen, Weihua He, Jiahao He, Yuanyu Wang, Fan Haffari, Gholamreza Zhuang, Bohan Computer Vision and Pattern Recognition Diffusion generative models have become the standard for producing high-quality, coherent video content, yet their slow inference speeds and high computational demands hinder practical deployment. Although both quantization and sparsity can independently accelerate inference while maintaining generation quality, naively combining these techniques in existing training-free approaches leads to significant performance degradation due to the lack of joint optimization. We introduce FPSAttention, a novel training-aware co-design of FP8 quantization and sparsity for video generation, with a focus on the 3D bi-directional attention mechanism. Our approach features three key innovations: 1) A unified 3D tile-wise granularity that simultaneously supports both quantization and sparsity; 2) A denoising step-aware strategy that adapts to the noise schedule, addressing the strong correlation between quantization/sparsity errors and denoising steps; 3) A native, hardware-friendly kernel that leverages FlashAttention and is implemented with optimized Hopper architecture features for highly efficient execution. Trained on Wan2.1's 1.3B and 14B models and evaluated on the VBench benchmark, FPSAttention achieves a 7.09x kernel speedup for attention operations and a 4.96x end-to-end speedup for video generation compared to the BF16 baseline at 720p resolution-without sacrificing generation quality. |
| title | FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.04648 |