Label Smoothing is a Pragmatic Information Bottleneck
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915452055191552 |
|---|---|
| author | Kudo, Sota |
| author_facet | Kudo, Sota |
| contents | This study revisits label smoothing via a form of information bottleneck. Under the assumption of sufficient model flexibility and no conflicting labels for the same input, we theoretically and experimentally demonstrate that the model output obtained through label smoothing explores the optimal solution of the information bottleneck. Based on this, label smoothing can be interpreted as a practical approach to the information bottleneck, enabling simple implementation. As an information bottleneck method, we experimentally show that label smoothing also exhibits the property of being insensitive to factors that do not contain information about the target, or to factors that provide no additional information about it when conditioned on another variable. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_14077 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Label Smoothing is a Pragmatic Information Bottleneck Kudo, Sota Machine Learning Artificial Intelligence 68T05 I.2.6; I.5.1 This study revisits label smoothing via a form of information bottleneck. Under the assumption of sufficient model flexibility and no conflicting labels for the same input, we theoretically and experimentally demonstrate that the model output obtained through label smoothing explores the optimal solution of the information bottleneck. Based on this, label smoothing can be interpreted as a practical approach to the information bottleneck, enabling simple implementation. As an information bottleneck method, we experimentally show that label smoothing also exhibits the property of being insensitive to factors that do not contain information about the target, or to factors that provide no additional information about it when conditioned on another variable. |
| title | Label Smoothing is a Pragmatic Information Bottleneck |
| topic | Machine Learning Artificial Intelligence 68T05 I.2.6; I.5.1 |
| url | https://arxiv.org/abs/2508.14077 |