PTSBench: A Comprehensive Post-Training Sparsity Benchmark Towards Algorithms and Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915056156934144 |
|---|---|
| author | Wnag, Zining Guo, Jinyang Gong, Ruihao Yong, Yang Liu, Aishan Huang, Yushi Liu, Jiaheng Liu, Xianglong |
| author_facet | Wnag, Zining Guo, Jinyang Gong, Ruihao Yong, Yang Liu, Aishan Huang, Yushi Liu, Jiaheng Liu, Xianglong |
| contents | With the increased attention to model efficiency, post-training sparsity (PTS) has become more and more prevalent because of its effectiveness and efficiency. However, there remain questions on better practice of PTS algorithms and the sparsification ability of models, which hinders the further development of this area. Therefore, a benchmark to comprehensively investigate the issues above is urgently needed. In this paper, we propose the first comprehensive post-training sparsity benchmark called PTSBench towards algorithms and models. We benchmark 10+ PTS general-pluggable fine-grained techniques on 3 typical tasks using over 40 off-the-shelf model architectures. Through extensive experiments and analyses, we obtain valuable conclusions and provide several insights from both algorithms and model aspects. Our PTSBench can provide (1) new observations for a better understanding of the PTS algorithms, (2) in-depth and comprehensive evaluations for the sparsification ability of models, and (3) a well-structured and easy-integrate open-source framework. We hope this work will provide illuminating conclusions and advice for future studies of post-training sparsity methods and sparsification-friendly model design. The code for our PTSBench is released at \href{https://github.com/ModelTC/msbench}{https://github.com/ModelTC/msbench}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_07268 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PTSBench: A Comprehensive Post-Training Sparsity Benchmark Towards Algorithms and Models Wnag, Zining Guo, Jinyang Gong, Ruihao Yong, Yang Liu, Aishan Huang, Yushi Liu, Jiaheng Liu, Xianglong Machine Learning Multimedia With the increased attention to model efficiency, post-training sparsity (PTS) has become more and more prevalent because of its effectiveness and efficiency. However, there remain questions on better practice of PTS algorithms and the sparsification ability of models, which hinders the further development of this area. Therefore, a benchmark to comprehensively investigate the issues above is urgently needed. In this paper, we propose the first comprehensive post-training sparsity benchmark called PTSBench towards algorithms and models. We benchmark 10+ PTS general-pluggable fine-grained techniques on 3 typical tasks using over 40 off-the-shelf model architectures. Through extensive experiments and analyses, we obtain valuable conclusions and provide several insights from both algorithms and model aspects. Our PTSBench can provide (1) new observations for a better understanding of the PTS algorithms, (2) in-depth and comprehensive evaluations for the sparsification ability of models, and (3) a well-structured and easy-integrate open-source framework. We hope this work will provide illuminating conclusions and advice for future studies of post-training sparsity methods and sparsification-friendly model design. The code for our PTSBench is released at \href{https://github.com/ModelTC/msbench}{https://github.com/ModelTC/msbench}. |
| title | PTSBench: A Comprehensive Post-Training Sparsity Benchmark Towards Algorithms and Models |
| topic | Machine Learning Multimedia |
| url | https://arxiv.org/abs/2412.07268 |