NM-SpMM: Accelerating Matrix Multiplication Using N:M Sparsity with GPGPU
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909523792363520 |
|---|---|
| author | Ma, Cong Wu, Du Deng, Zhelang Chen, Jiang Huang, Xiaowen Meng, Jintao Zhu, Wenxi Wang, Bingqiang Zhou, Amelie Chi Chen, Peng Deng, Minwen Wei, Yanjie Feng, Shengzhong Pan, Yi |
| author_facet | Ma, Cong Wu, Du Deng, Zhelang Chen, Jiang Huang, Xiaowen Meng, Jintao Zhu, Wenxi Wang, Bingqiang Zhou, Amelie Chi Chen, Peng Deng, Minwen Wei, Yanjie Feng, Shengzhong Pan, Yi |
| contents | Deep learning demonstrates effectiveness across a wide range of tasks. However, the dense and over-parameterized nature of these models results in significant resource consumption during deployment. In response to this issue, weight pruning, particularly through N:M sparsity matrix multiplication, offers an efficient solution by transforming dense operations into semi-sparse ones. N:M sparsity provides an option for balancing performance and model accuracy, but introduces more complex programming and optimization challenges. To address these issues, we design a systematic top-down performance analysis model for N:M sparsity. Meanwhile, NM-SpMM is proposed as an efficient general N:M sparsity implementation. Based on our performance analysis, NM-SpMM employs a hierarchical blocking mechanism as a general optimization to enhance data locality, while memory access optimization and pipeline design are introduced as sparsity-aware optimization, allowing it to achieve close-to-theoretical peak performance across different sparsity levels. Experimental results show that NM-SpMM is 2.1x faster than nmSPARSE (the state-of-the-art for general N:M sparsity) and 1.4x to 6.3x faster than cuBLAS's dense GEMM operations, closely approaching the theoretical maximum speedup resulting from the reduction in computation due to sparsity. NM-SpMM is open source and publicly available at https://github.com/M-H482/NM-SpMM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_01253 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | NM-SpMM: Accelerating Matrix Multiplication Using N:M Sparsity with GPGPU Ma, Cong Wu, Du Deng, Zhelang Chen, Jiang Huang, Xiaowen Meng, Jintao Zhu, Wenxi Wang, Bingqiang Zhou, Amelie Chi Chen, Peng Deng, Minwen Wei, Yanjie Feng, Shengzhong Pan, Yi Distributed, Parallel, and Cluster Computing C.1.4; D.1.3; G.1.0 Deep learning demonstrates effectiveness across a wide range of tasks. However, the dense and over-parameterized nature of these models results in significant resource consumption during deployment. In response to this issue, weight pruning, particularly through N:M sparsity matrix multiplication, offers an efficient solution by transforming dense operations into semi-sparse ones. N:M sparsity provides an option for balancing performance and model accuracy, but introduces more complex programming and optimization challenges. To address these issues, we design a systematic top-down performance analysis model for N:M sparsity. Meanwhile, NM-SpMM is proposed as an efficient general N:M sparsity implementation. Based on our performance analysis, NM-SpMM employs a hierarchical blocking mechanism as a general optimization to enhance data locality, while memory access optimization and pipeline design are introduced as sparsity-aware optimization, allowing it to achieve close-to-theoretical peak performance across different sparsity levels. Experimental results show that NM-SpMM is 2.1x faster than nmSPARSE (the state-of-the-art for general N:M sparsity) and 1.4x to 6.3x faster than cuBLAS's dense GEMM operations, closely approaching the theoretical maximum speedup resulting from the reduction in computation due to sparsity. NM-SpMM is open source and publicly available at https://github.com/M-H482/NM-SpMM. |
| title | NM-SpMM: Accelerating Matrix Multiplication Using N:M Sparsity with GPGPU |
| topic | Distributed, Parallel, and Cluster Computing C.1.4; D.1.3; G.1.0 |
| url | https://arxiv.org/abs/2503.01253 |