Mix-and-Match Pruning: Globally Guided Layer-Wise Sparsification of DNNs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917355300323328 |
|---|---|
| author | Monachan, Danial Nazari, Samira Taheri, Mahdi Azarpeyvand, Ali Krstic, Milos Huebner, Michael Herglotz, Christian |
| author_facet | Monachan, Danial Nazari, Samira Taheri, Mahdi Azarpeyvand, Ali Krstic, Milos Huebner, Michael Herglotz, Christian |
| contents | Deploying deep neural networks (DNNs) on edge devices requires strong compression with minimal accuracy loss. This paper introduces Mix-and-Match Pruning, a globally guided, layer-wise sparsification framework that leverages sensitivity scores and simple architectural rules to generate diverse, high-quality pruning configurations. The framework addresses a key limitation that different layers and architectures respond differently to pruning, making single-strategy approaches suboptimal. Mix-and-Match derives architecture-aware sparsity ranges, e.g., preserving normalization layers while pruning classifiers more aggressively, and systematically samples these ranges to produce ten strategies per sensitivity signal (magnitude, gradient, or their combination). This eliminates repeated pruning runs while offering deployment-ready accuracy-sparsity trade-offs. Experiments on CNNs and Vision Transformers demonstrate Pareto-optimal results, with Mix-and-Match reducing accuracy degradation on Swin-Tiny by 40% relative to standard single-criterion pruning. These findings show that coordinating existing pruning signals enables more reliable and efficient compressed models than introducing new criteria. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_20280 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Mix-and-Match Pruning: Globally Guided Layer-Wise Sparsification of DNNs Monachan, Danial Nazari, Samira Taheri, Mahdi Azarpeyvand, Ali Krstic, Milos Huebner, Michael Herglotz, Christian Computer Vision and Pattern Recognition Hardware Architecture Machine Learning Deploying deep neural networks (DNNs) on edge devices requires strong compression with minimal accuracy loss. This paper introduces Mix-and-Match Pruning, a globally guided, layer-wise sparsification framework that leverages sensitivity scores and simple architectural rules to generate diverse, high-quality pruning configurations. The framework addresses a key limitation that different layers and architectures respond differently to pruning, making single-strategy approaches suboptimal. Mix-and-Match derives architecture-aware sparsity ranges, e.g., preserving normalization layers while pruning classifiers more aggressively, and systematically samples these ranges to produce ten strategies per sensitivity signal (magnitude, gradient, or their combination). This eliminates repeated pruning runs while offering deployment-ready accuracy-sparsity trade-offs. Experiments on CNNs and Vision Transformers demonstrate Pareto-optimal results, with Mix-and-Match reducing accuracy degradation on Swin-Tiny by 40% relative to standard single-criterion pruning. These findings show that coordinating existing pruning signals enables more reliable and efficient compressed models than introducing new criteria. |
| title | Mix-and-Match Pruning: Globally Guided Layer-Wise Sparsification of DNNs |
| topic | Computer Vision and Pattern Recognition Hardware Architecture Machine Learning |
| url | https://arxiv.org/abs/2603.20280 |