Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.07119 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912267500519424 |
|---|---|
| author | Kuzin, Danil Isupova, Olga Reece, Steven Simmons, Brooke D |
| author_facet | Kuzin, Danil Isupova, Olga Reece, Steven Simmons, Brooke D |
| contents | Ensembling in deep learning improves accuracy and calibration over single networks. The traditional aggregation approach, ensemble averaging, treats all individual networks equally by averaging their outputs. Inspired by crowdsourcing we propose an aggregation method called soft Dawid Skene for deep ensembles that estimates confusion matrices of ensemble members and weighs them according to their inferred performance. Soft Dawid Skene aggregates soft labels in contrast to hard labels often used in crowdsourcing. We empirically show the superiority of soft Dawid Skene in accuracy, calibration and out of distribution detection in comparison to ensemble averaging in extensive experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_07119 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Improving Deep Ensembles by Estimating Confusion Matrices Kuzin, Danil Isupova, Olga Reece, Steven Simmons, Brooke D Machine Learning Ensembling in deep learning improves accuracy and calibration over single networks. The traditional aggregation approach, ensemble averaging, treats all individual networks equally by averaging their outputs. Inspired by crowdsourcing we propose an aggregation method called soft Dawid Skene for deep ensembles that estimates confusion matrices of ensemble members and weighs them according to their inferred performance. Soft Dawid Skene aggregates soft labels in contrast to hard labels often used in crowdsourcing. We empirically show the superiority of soft Dawid Skene in accuracy, calibration and out of distribution detection in comparison to ensemble averaging in extensive experiments. |
| title | Improving Deep Ensembles by Estimating Confusion Matrices |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2503.07119 |