Variational Learning Induces Adaptive Label Smoothing
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913717617164288 |
|---|---|
| author | Yang, Sin-Han Liu, Zhedong Marconi, Gian Maria Khan, Mohammad Emtiyaz |
| author_facet | Yang, Sin-Han Liu, Zhedong Marconi, Gian Maria Khan, Mohammad Emtiyaz |
| contents | We show that variational learning naturally induces an adaptive label smoothing where label noise is specialized for each example. Such label-smoothing is useful to handle examples with labeling errors and distribution shifts, but designing a good adaptivity strategy is not always easy. We propose to skip this step and simply use the natural adaptivity induced during the optimization of a variational objective. We show empirical results where a variational algorithm called IVON outperforms traditional label smoothing and yields adaptivity strategies similar to those of an existing approach. By connecting Bayesian methods to label smoothing, our work provides a new way to handle overconfident predictions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_07273 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Variational Learning Induces Adaptive Label Smoothing Yang, Sin-Han Liu, Zhedong Marconi, Gian Maria Khan, Mohammad Emtiyaz Machine Learning Artificial Intelligence We show that variational learning naturally induces an adaptive label smoothing where label noise is specialized for each example. Such label-smoothing is useful to handle examples with labeling errors and distribution shifts, but designing a good adaptivity strategy is not always easy. We propose to skip this step and simply use the natural adaptivity induced during the optimization of a variational objective. We show empirical results where a variational algorithm called IVON outperforms traditional label smoothing and yields adaptivity strategies similar to those of an existing approach. By connecting Bayesian methods to label smoothing, our work provides a new way to handle overconfident predictions. |
| title | Variational Learning Induces Adaptive Label Smoothing |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2502.07273 |