Sharp Minima Can Generalize: A Loss Landscape Perspective On Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912692866908160 |
|---|---|
| author | Fan, Raymond Sandlund, Bryce Ko, Lin Myat |
| author_facet | Fan, Raymond Sandlund, Bryce Ko, Lin Myat |
| contents | The volume hypothesis suggests deep learning is effective because it is likely to find flat minima due to their large volumes, and flat minima generalize well. This picture does not explain the role of large datasets in generalization. Measuring minima volumes under varying amounts of training data reveals sharp minima which generalize well exist, but are unlikely to be found due to their small volumes. Increasing data changes the loss landscape, such that previously small generalizing minima become (relatively) large. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_04808 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Sharp Minima Can Generalize: A Loss Landscape Perspective On Data Fan, Raymond Sandlund, Bryce Ko, Lin Myat Machine Learning The volume hypothesis suggests deep learning is effective because it is likely to find flat minima due to their large volumes, and flat minima generalize well. This picture does not explain the role of large datasets in generalization. Measuring minima volumes under varying amounts of training data reveals sharp minima which generalize well exist, but are unlikely to be found due to their small volumes. Increasing data changes the loss landscape, such that previously small generalizing minima become (relatively) large. |
| title | Sharp Minima Can Generalize: A Loss Landscape Perspective On Data |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2511.04808 |