Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.12827 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917442080473088 |
|---|---|
| author | Kim, Taeyoung |
| author_facet | Kim, Taeyoung |
| contents | We investigate random feature models in which neural networks sampled from a prescribed initialization ensemble are frozen and used as random features, with only the readout weights optimized. Adopting a statistical-physics viewpoint, we study the training error, test error, and generalization gap beyond the mean kernel approximation. Since the predictor is a nonlinear functional of the induced random kernel, the ensemble-averaged errors depend not only on the mean kernel but also on higher-order fluctuation statistics. Within an effective field-theoretic framework, these finite-width contributions naturally appear as loop corrections. We derive loop corrections to the training error, test error, and generalization gap, obtain their scaling laws, and support the theory with experimental verification. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_12827 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Loop Corrections to the Training Error and Generalization Gap of Random Feature Models Kim, Taeyoung Machine Learning Artificial Intelligence We investigate random feature models in which neural networks sampled from a prescribed initialization ensemble are frozen and used as random features, with only the readout weights optimized. Adopting a statistical-physics viewpoint, we study the training error, test error, and generalization gap beyond the mean kernel approximation. Since the predictor is a nonlinear functional of the induced random kernel, the ensemble-averaged errors depend not only on the mean kernel but also on higher-order fluctuation statistics. Within an effective field-theoretic framework, these finite-width contributions naturally appear as loop corrections. We derive loop corrections to the training error, test error, and generalization gap, obtain their scaling laws, and support the theory with experimental verification. |
| title | Loop Corrections to the Training Error and Generalization Gap of Random Feature Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2604.12827 |