The Simpler The Better: An Entropy-Based Importance Metric To Reduce Neural Networks' Depth
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913377484275712 |
|---|---|
| author | Quétu, Victor Liao, Zhu Tartaglione, Enzo |
| author_facet | Quétu, Victor Liao, Zhu Tartaglione, Enzo |
| contents | While deep neural networks are highly effective at solving complex tasks, large pre-trained models are commonly employed even to solve consistently simpler downstream tasks, which do not necessarily require a large model's complexity. Motivated by the awareness of the ever-growing AI environmental impact, we propose an efficiency strategy that leverages prior knowledge transferred by large models. Simple but effective, we propose a method relying on an Entropy-bASed Importance mEtRic (EASIER) to reduce the depth of over-parametrized deep neural networks, which alleviates their computational burden. We assess the effectiveness of our method on traditional image classification setups. Our code is available at https://github.com/VGCQ/EASIER. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_18949 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Simpler The Better: An Entropy-Based Importance Metric To Reduce Neural Networks' Depth Quétu, Victor Liao, Zhu Tartaglione, Enzo Machine Learning While deep neural networks are highly effective at solving complex tasks, large pre-trained models are commonly employed even to solve consistently simpler downstream tasks, which do not necessarily require a large model's complexity. Motivated by the awareness of the ever-growing AI environmental impact, we propose an efficiency strategy that leverages prior knowledge transferred by large models. Simple but effective, we propose a method relying on an Entropy-bASed Importance mEtRic (EASIER) to reduce the depth of over-parametrized deep neural networks, which alleviates their computational burden. We assess the effectiveness of our method on traditional image classification setups. Our code is available at https://github.com/VGCQ/EASIER. |
| title | The Simpler The Better: An Entropy-Based Importance Metric To Reduce Neural Networks' Depth |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2404.18949 |