Select High-Level Features: Efficient Experts from a Hierarchical Classification Network
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915026863915008 |
|---|---|
| author | Kelm, André Hannemann, Niels Heberle, Bruno Schmidt, Lucas Rolff, Tim Wilms, Christian Yaghoubi, Ehsan Frintrop, Simone |
| author_facet | Kelm, André Hannemann, Niels Heberle, Bruno Schmidt, Lucas Rolff, Tim Wilms, Christian Yaghoubi, Ehsan Frintrop, Simone |
| contents | This study introduces a novel expert generation method that dynamically reduces task and computational complexity without compromising predictive performance. It is based on a new hierarchical classification network topology that combines sequential processing of generic low-level features with parallelism and nesting of high-level features. This structure allows for the innovative extraction technique: the ability to select only high-level features of task-relevant categories. In certain cases, it is possible to skip almost all unneeded high-level features, which can significantly reduce the inference cost and is highly beneficial in resource-constrained conditions. We believe this method paves the way for future network designs that are lightweight and adaptable, making them suitable for a wide range of applications, from compact edge devices to large-scale clouds. In terms of dynamic inference our methodology can achieve an exclusion of up to 88.7\,\% of parameters and 73.4\,\% fewer giga-multiply accumulate (GMAC) operations, analysis against comparative baselines showing an average reduction of 47.6\,\% in parameters and 5.8\,\% in GMACs across the cases we evaluated. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_05601 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Select High-Level Features: Efficient Experts from a Hierarchical Classification Network Kelm, André Hannemann, Niels Heberle, Bruno Schmidt, Lucas Rolff, Tim Wilms, Christian Yaghoubi, Ehsan Frintrop, Simone Machine Learning This study introduces a novel expert generation method that dynamically reduces task and computational complexity without compromising predictive performance. It is based on a new hierarchical classification network topology that combines sequential processing of generic low-level features with parallelism and nesting of high-level features. This structure allows for the innovative extraction technique: the ability to select only high-level features of task-relevant categories. In certain cases, it is possible to skip almost all unneeded high-level features, which can significantly reduce the inference cost and is highly beneficial in resource-constrained conditions. We believe this method paves the way for future network designs that are lightweight and adaptable, making them suitable for a wide range of applications, from compact edge devices to large-scale clouds. In terms of dynamic inference our methodology can achieve an exclusion of up to 88.7\,\% of parameters and 73.4\,\% fewer giga-multiply accumulate (GMAC) operations, analysis against comparative baselines showing an average reduction of 47.6\,\% in parameters and 5.8\,\% in GMACs across the cases we evaluated. |
| title | Select High-Level Features: Efficient Experts from a Hierarchical Classification Network |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2403.05601 |