Phase Transitions in Neural Networks Pruning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917277740302336 |
|---|---|
| author | Pesce, Diego He, Yang-Hui Caldarelli, Guido |
| author_facet | Pesce, Diego He, Yang-Hui Caldarelli, Guido |
| contents | Deep neural networks are strongly over-parameterized, often containing far more weights than required for their task. Although such redundancy can aid optimization, it leads to inefficient deployment and high computational cost, motivating model compression techniques. Among these, network pruning provides a clear and effective route to sparsity. We study pruning from a statistical-physics perspective, interpreting performance degradation under weight removal as a phase transition. Focusing on magnitude-based pruning with fine-tuning, we show that deep networks undergo a sharp transition from a cooperative, functional phase to a disordered phase with collapsed performance. This transition is characterized by scaling laws consistent with second-order critical behavior, with connectivity as the control parameter. Our findings suggest universal pruning-induced criticality across architectures and datasets. Finally, we show that there exists a large class of subnetworks sharing the same nodes' degrees with similar learning ability, thus linking model performance to its topological properties. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15224 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Phase Transitions in Neural Networks Pruning Pesce, Diego He, Yang-Hui Caldarelli, Guido Disordered Systems and Neural Networks Adaptation and Self-Organizing Systems Deep neural networks are strongly over-parameterized, often containing far more weights than required for their task. Although such redundancy can aid optimization, it leads to inefficient deployment and high computational cost, motivating model compression techniques. Among these, network pruning provides a clear and effective route to sparsity. We study pruning from a statistical-physics perspective, interpreting performance degradation under weight removal as a phase transition. Focusing on magnitude-based pruning with fine-tuning, we show that deep networks undergo a sharp transition from a cooperative, functional phase to a disordered phase with collapsed performance. This transition is characterized by scaling laws consistent with second-order critical behavior, with connectivity as the control parameter. Our findings suggest universal pruning-induced criticality across architectures and datasets. Finally, we show that there exists a large class of subnetworks sharing the same nodes' degrees with similar learning ability, thus linking model performance to its topological properties. |
| title | Phase Transitions in Neural Networks Pruning |
| topic | Disordered Systems and Neural Networks Adaptation and Self-Organizing Systems |
| url | https://arxiv.org/abs/2602.15224 |