Optimal Brain Connection: Towards Efficient Structural Pruning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911096118444032 |
|---|---|
| author | Chen, Shaowu Ma, Wei Huang, Binhua Wang, Qingyuan Wang, Guoxin Sun, Weize Huang, Lei John, Deepu |
| author_facet | Chen, Shaowu Ma, Wei Huang, Binhua Wang, Qingyuan Wang, Guoxin Sun, Weize Huang, Lei John, Deepu |
| contents | Structural pruning has been widely studied for its effectiveness in compressing neural networks. However, existing methods often neglect the interconnections among parameters. To address this limitation, this paper proposes a structural pruning framework termed Optimal Brain Connection. First, we introduce the Jacobian Criterion, a first-order metric for evaluating the saliency of structural parameters. Unlike existing first-order methods that assess parameters in isolation, our criterion explicitly captures both intra-component interactions and inter-layer dependencies. Second, we propose the Equivalent Pruning mechanism, which utilizes autoencoders to retain the contributions of all original connection--including pruned ones--during fine-tuning. Experimental results demonstrate that the Jacobian Criterion outperforms several popular metrics in preserving model performance, while the Equivalent Pruning mechanism effectively mitigates performance degradation after fine-tuning. Code: https://github.com/ShaowuChen/Optimal_Brain_Connection |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_05521 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Optimal Brain Connection: Towards Efficient Structural Pruning Chen, Shaowu Ma, Wei Huang, Binhua Wang, Qingyuan Wang, Guoxin Sun, Weize Huang, Lei John, Deepu Computer Vision and Pattern Recognition Structural pruning has been widely studied for its effectiveness in compressing neural networks. However, existing methods often neglect the interconnections among parameters. To address this limitation, this paper proposes a structural pruning framework termed Optimal Brain Connection. First, we introduce the Jacobian Criterion, a first-order metric for evaluating the saliency of structural parameters. Unlike existing first-order methods that assess parameters in isolation, our criterion explicitly captures both intra-component interactions and inter-layer dependencies. Second, we propose the Equivalent Pruning mechanism, which utilizes autoencoders to retain the contributions of all original connection--including pruned ones--during fine-tuning. Experimental results demonstrate that the Jacobian Criterion outperforms several popular metrics in preserving model performance, while the Equivalent Pruning mechanism effectively mitigates performance degradation after fine-tuning. Code: https://github.com/ShaowuChen/Optimal_Brain_Connection |
| title | Optimal Brain Connection: Towards Efficient Structural Pruning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.05521 |