Confidence-gated training for efficient early-exit neural networks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912810485678080 |
|---|---|
| author | Mokssit, Saad Karrakchou, Ouassim Mousist, Alejandro Ghogho, Mounir |
| author_facet | Mokssit, Saad Karrakchou, Ouassim Mousist, Alejandro Ghogho, Mounir |
| contents | Early-exit neural networks reduce inference cost by enabling confident predictions at intermediate layers. However, joint training often leads to gradient interference, with deeper classifiers dominating optimization. We propose Confidence-Gated Training (CGT), a paradigm that conditionally propagates gradients from deeper exits only when preceding exits fail. This encourages shallow classifiers to act as primary decision points while reserving deeper layers for harder inputs. By aligning training with the inference-time policy, CGT mitigates overthinking, improves early-exit accuracy, and preserves efficiency. Experiments on the Indian Pines and Fashion-MNIST benchmarks show that CGT lowers average inference cost while improving overall accuracy, offering a practical solution for deploying deep models in resource-constrained environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_17885 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Confidence-gated training for efficient early-exit neural networks Mokssit, Saad Karrakchou, Ouassim Mousist, Alejandro Ghogho, Mounir Machine Learning Artificial Intelligence Early-exit neural networks reduce inference cost by enabling confident predictions at intermediate layers. However, joint training often leads to gradient interference, with deeper classifiers dominating optimization. We propose Confidence-Gated Training (CGT), a paradigm that conditionally propagates gradients from deeper exits only when preceding exits fail. This encourages shallow classifiers to act as primary decision points while reserving deeper layers for harder inputs. By aligning training with the inference-time policy, CGT mitigates overthinking, improves early-exit accuracy, and preserves efficiency. Experiments on the Indian Pines and Fashion-MNIST benchmarks show that CGT lowers average inference cost while improving overall accuracy, offering a practical solution for deploying deep models in resource-constrained environments. |
| title | Confidence-gated training for efficient early-exit neural networks |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2509.17885 |