Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.16936 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914275712303104 |
|---|---|
| author | Zamyatin, Anton Indri, Patrick Malhotra, Sagar Gärtner, Thomas |
| author_facet | Zamyatin, Anton Indri, Patrick Malhotra, Sagar Gärtner, Thomas |
| contents | In resource-constrained and low-latency settings, uncertainty estimates must be efficiently obtained. Deep Ensembles provide robust epistemic uncertainty (EU) but require training multiple full-size models. BatchEnsemble aims to deliver ensemble-like EU at far lower parameter and memory cost by applying learned rank-1 perturbations to a shared base network. We show that BatchEnsemble not only underperforms Deep Ensembles but closely tracks a single model baseline in terms of accuracy, calibration and out-of-distribution (OOD) detection on CIFAR10/10C/SVHN. A controlled study on MNIST finds members are near-identical in function and parameter space, indicating limited capacity to realize distinct predictive modes. Thus, BatchEnsemble behaves more like a single model than a true ensemble. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_16936 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Is BatchEnsemble a Single Model? On Calibration and Diversity of Efficient Ensembles Zamyatin, Anton Indri, Patrick Malhotra, Sagar Gärtner, Thomas Machine Learning In resource-constrained and low-latency settings, uncertainty estimates must be efficiently obtained. Deep Ensembles provide robust epistemic uncertainty (EU) but require training multiple full-size models. BatchEnsemble aims to deliver ensemble-like EU at far lower parameter and memory cost by applying learned rank-1 perturbations to a shared base network. We show that BatchEnsemble not only underperforms Deep Ensembles but closely tracks a single model baseline in terms of accuracy, calibration and out-of-distribution (OOD) detection on CIFAR10/10C/SVHN. A controlled study on MNIST finds members are near-identical in function and parameter space, indicating limited capacity to realize distinct predictive modes. Thus, BatchEnsemble behaves more like a single model than a true ensemble. |
| title | Is BatchEnsemble a Single Model? On Calibration and Diversity of Efficient Ensembles |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2601.16936 |