Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912403581566976 |
|---|---|
| author | Tan, Chengli Zhou, Yubo Ye, Haishan Dai, Guang Liu, Junmin Song, Zengjie Zhang, Jiangshe Zhao, Zixiang Hao, Yunda Xu, Yong |
| author_facet | Tan, Chengli Zhou, Yubo Ye, Haishan Dai, Guang Liu, Junmin Song, Zengjie Zhang, Jiangshe Zhao, Zixiang Hao, Yunda Xu, Yong |
| contents | Deep neural networks have been increasingly used in safety-critical applications such as medical diagnosis and autonomous driving. However, many studies suggest that they are prone to being poorly calibrated and have a propensity for overconfidence, which may have disastrous consequences. In this paper, unlike standard training such as stochastic gradient descent, we show that the recently proposed sharpness-aware minimization (SAM) counteracts this tendency towards overconfidence. The theoretical analysis suggests that SAM allows us to learn models that are already well-calibrated by implicitly maximizing the entropy of the predictive distribution. Inspired by this finding, we further propose a variant of SAM, coined as CSAM, to ameliorate model calibration. Extensive experiments on various datasets, including ImageNet-1K, demonstrate the benefits of SAM in reducing calibration error. Meanwhile, CSAM performs even better than SAM and consistently achieves lower calibration error than other approaches |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_23866 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization Tan, Chengli Zhou, Yubo Ye, Haishan Dai, Guang Liu, Junmin Song, Zengjie Zhang, Jiangshe Zhao, Zixiang Hao, Yunda Xu, Yong Machine Learning Artificial Intelligence Deep neural networks have been increasingly used in safety-critical applications such as medical diagnosis and autonomous driving. However, many studies suggest that they are prone to being poorly calibrated and have a propensity for overconfidence, which may have disastrous consequences. In this paper, unlike standard training such as stochastic gradient descent, we show that the recently proposed sharpness-aware minimization (SAM) counteracts this tendency towards overconfidence. The theoretical analysis suggests that SAM allows us to learn models that are already well-calibrated by implicitly maximizing the entropy of the predictive distribution. Inspired by this finding, we further propose a variant of SAM, coined as CSAM, to ameliorate model calibration. Extensive experiments on various datasets, including ImageNet-1K, demonstrate the benefits of SAM in reducing calibration error. Meanwhile, CSAM performs even better than SAM and consistently achieves lower calibration error than other approaches |
| title | Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2505.23866 |