CFSP: An Efficient Structured Pruning Framework for LLMs with Coarse-to-Fine Activation Information
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910735523643392 |
|---|---|
| author | Wang, Yuxin Ma, Minghua Wang, Zekun Chen, Jingchang Fan, Huiming Shan, Liping Yang, Qing Xu, Dongliang Liu, Ming Qin, Bing |
| author_facet | Wang, Yuxin Ma, Minghua Wang, Zekun Chen, Jingchang Fan, Huiming Shan, Liping Yang, Qing Xu, Dongliang Liu, Ming Qin, Bing |
| contents | The colossal parameters and computational overhead of Large Language Models (LLMs) challenge their real-world applications. Network pruning, which targets unstructured or structured sparsity by removing redundant parameters, has recently been explored for LLM acceleration. Existing LLM pruning works focus on unstructured pruning, which typically requires special hardware support for a practical speed-up. In contrast, structured pruning can reduce latency on general devices. However, it remains a challenge to perform structured pruning efficiently and maintain performance, especially at high sparsity ratios. To this end, we introduce an efficient structured pruning framework named CFSP, which leverages both Coarse (interblock) and Fine-grained (intrablock) activation information as an importance criterion to guide pruning. The pruning is highly efficient, as it only requires one forward pass to compute feature activations. Specifically, we first allocate the sparsity budget across blocks based on their importance and then retain important weights within each block. In addition, we introduce a recovery fine-tuning strategy that adaptively allocates training overhead based on coarse-grained importance to further improve performance. Experimental results demonstrate that CFSP outperforms existing methods on diverse models across various sparsity budgets. Our code will be available at https://github.com/wyxscir/CFSP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_13199 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CFSP: An Efficient Structured Pruning Framework for LLMs with Coarse-to-Fine Activation Information Wang, Yuxin Ma, Minghua Wang, Zekun Chen, Jingchang Fan, Huiming Shan, Liping Yang, Qing Xu, Dongliang Liu, Ming Qin, Bing Computation and Language The colossal parameters and computational overhead of Large Language Models (LLMs) challenge their real-world applications. Network pruning, which targets unstructured or structured sparsity by removing redundant parameters, has recently been explored for LLM acceleration. Existing LLM pruning works focus on unstructured pruning, which typically requires special hardware support for a practical speed-up. In contrast, structured pruning can reduce latency on general devices. However, it remains a challenge to perform structured pruning efficiently and maintain performance, especially at high sparsity ratios. To this end, we introduce an efficient structured pruning framework named CFSP, which leverages both Coarse (interblock) and Fine-grained (intrablock) activation information as an importance criterion to guide pruning. The pruning is highly efficient, as it only requires one forward pass to compute feature activations. Specifically, we first allocate the sparsity budget across blocks based on their importance and then retain important weights within each block. In addition, we introduce a recovery fine-tuning strategy that adaptively allocates training overhead based on coarse-grained importance to further improve performance. Experimental results demonstrate that CFSP outperforms existing methods on diverse models across various sparsity budgets. Our code will be available at https://github.com/wyxscir/CFSP. |
| title | CFSP: An Efficient Structured Pruning Framework for LLMs with Coarse-to-Fine Activation Information |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.13199 |