Advanced deep architecture pruning using single filter performance
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915338594025472 |
|---|---|
| author | Tzach, Yarden Meir, Yuval Gross, Ronit D. Tevet, Ofek Koresh, Ella Kanter, Ido |
| author_facet | Tzach, Yarden Meir, Yuval Gross, Ronit D. Tevet, Ofek Koresh, Ella Kanter, Ido |
| contents | Pruning the parameters and structure of neural networks reduces the computational complexity, energy consumption, and latency during inference. Recently, a novel underlying mechanism for successful deep learning (DL) was presented based on a method that quantitatively measures the single filter performance in each layer of a DL architecture, and a new comprehensive mechanism of how deep learning works was presented. This statistical mechanics inspired viewpoint enables to reveal the macroscopic behavior of the entire network from the microscopic performance of each filter and their cooperative behavior. Herein, we demonstrate how this understanding paves the path to high quenched dilution of the convolutional layers of deep architectures without affecting their overall accuracy using applied filter cluster connections (AFCC). AFCC is exemplified on VGG-11 and EfficientNet-B0 architectures trained on CIFAR-100, and its high pruning outperforms other techniques using the same pruning magnitude. Additionally, this technique is broadened to single nodal performance and highly pruning of fully connected layers, suggesting a possible implementation to considerably reduce the complexity of over-parameterized AI tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_12880 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Advanced deep architecture pruning using single filter performance Tzach, Yarden Meir, Yuval Gross, Ronit D. Tevet, Ofek Koresh, Ella Kanter, Ido Machine Learning Computer Vision and Pattern Recognition Pruning the parameters and structure of neural networks reduces the computational complexity, energy consumption, and latency during inference. Recently, a novel underlying mechanism for successful deep learning (DL) was presented based on a method that quantitatively measures the single filter performance in each layer of a DL architecture, and a new comprehensive mechanism of how deep learning works was presented. This statistical mechanics inspired viewpoint enables to reveal the macroscopic behavior of the entire network from the microscopic performance of each filter and their cooperative behavior. Herein, we demonstrate how this understanding paves the path to high quenched dilution of the convolutional layers of deep architectures without affecting their overall accuracy using applied filter cluster connections (AFCC). AFCC is exemplified on VGG-11 and EfficientNet-B0 architectures trained on CIFAR-100, and its high pruning outperforms other techniques using the same pruning magnitude. Additionally, this technique is broadened to single nodal performance and highly pruning of fully connected layers, suggesting a possible implementation to considerably reduce the complexity of over-parameterized AI tasks. |
| title | Advanced deep architecture pruning using single filter performance |
| topic | Machine Learning Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.12880 |