Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916534050357248 |
|---|---|
| author | Liao, Zhu Hezbri, Nour Quétu, Victor Nguyen, Van-Tam Tartaglione, Enzo |
| author_facet | Liao, Zhu Hezbri, Nour Quétu, Victor Nguyen, Van-Tam Tartaglione, Enzo |
| contents | Today, deep neural networks are widely used since they can handle a variety of complex tasks. Their generality makes them very powerful tools in modern technology. However, deep neural networks are often overparameterized. The usage of these large models consumes a lot of computation resources. In this paper, we introduce a method called \textbf{T}ill the \textbf{L}ayers \textbf{C}ollapse (TLC), which compresses deep neural networks through the lenses of batch normalization layers. By reducing the depth of these networks, our method decreases deep neural networks' computational requirements and overall latency. We validate our method on popular models such as Swin-T, MobileNet-V2, and RoBERTa, across both image classification and natural language processing (NLP) tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15077 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers Liao, Zhu Hezbri, Nour Quétu, Victor Nguyen, Van-Tam Tartaglione, Enzo Machine Learning Computation and Language Computer Vision and Pattern Recognition Today, deep neural networks are widely used since they can handle a variety of complex tasks. Their generality makes them very powerful tools in modern technology. However, deep neural networks are often overparameterized. The usage of these large models consumes a lot of computation resources. In this paper, we introduce a method called \textbf{T}ill the \textbf{L}ayers \textbf{C}ollapse (TLC), which compresses deep neural networks through the lenses of batch normalization layers. By reducing the depth of these networks, our method decreases deep neural networks' computational requirements and overall latency. We validate our method on popular models such as Swin-T, MobileNet-V2, and RoBERTa, across both image classification and natural language processing (NLP) tasks. |
| title | Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers |
| topic | Machine Learning Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.15077 |