Feature learning is decoupled from generalization in high capacity neural networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909706390339584 |
|---|---|
| author | Göring, Niclas Alexander London, Charles Erturk, Abdurrahman Hadi Mingard, Chris Nam, Yoonsoo Louis, Ard A. |
| author_facet | Göring, Niclas Alexander London, Charles Erturk, Abdurrahman Hadi Mingard, Chris Nam, Yoonsoo Louis, Ard A. |
| contents | Neural networks outperform kernel methods, sometimes by orders of magnitude, e.g. on staircase functions. This advantage stems from the ability of neural networks to learn features, adapting their hidden representations to better capture the data. We introduce a concept we call feature quality to measure this performance improvement. We examine existing theories of feature learning and demonstrate empirically that they primarily assess the strength of feature learning, rather than the quality of the learned features themselves. Consequently, current theories of feature learning do not provide a sufficient foundation for developing theories of neural network generalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_19680 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Feature learning is decoupled from generalization in high capacity neural networks Göring, Niclas Alexander London, Charles Erturk, Abdurrahman Hadi Mingard, Chris Nam, Yoonsoo Louis, Ard A. Machine Learning Neural networks outperform kernel methods, sometimes by orders of magnitude, e.g. on staircase functions. This advantage stems from the ability of neural networks to learn features, adapting their hidden representations to better capture the data. We introduce a concept we call feature quality to measure this performance improvement. We examine existing theories of feature learning and demonstrate empirically that they primarily assess the strength of feature learning, rather than the quality of the learned features themselves. Consequently, current theories of feature learning do not provide a sufficient foundation for developing theories of neural network generalization. |
| title | Feature learning is decoupled from generalization in high capacity neural networks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2507.19680 |