Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.09184 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909670681083904 |
|---|---|
| author | Datres, Massimiliano Leonardi, Gian Paolo Figalli, Alessio Sutter, David |
| author_facet | Datres, Massimiliano Leonardi, Gian Paolo Figalli, Alessio Sutter, David |
| contents | We introduce a novel capacity measure 2sED for statistical models based on the effective dimension. The new quantity provably bounds the generalization error under mild assumptions on the model. Furthermore, simulations on standard data sets and popular model architectures show that 2sED correlates well with the training error. For Markovian models, we show how to efficiently approximate 2sED from below through a layerwise iterative approach, which allows us to tackle deep learning models with a large number of parameters. Simulation results suggest that the approximation is good for different prominent models and data sets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_09184 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Two-Scale Complexity Measure for Deep Learning Models Datres, Massimiliano Leonardi, Gian Paolo Figalli, Alessio Sutter, David Machine Learning We introduce a novel capacity measure 2sED for statistical models based on the effective dimension. The new quantity provably bounds the generalization error under mild assumptions on the model. Furthermore, simulations on standard data sets and popular model architectures show that 2sED correlates well with the training error. For Markovian models, we show how to efficiently approximate 2sED from below through a layerwise iterative approach, which allows us to tackle deep learning models with a large number of parameters. Simulation results suggest that the approximation is good for different prominent models and data sets. |
| title | A Two-Scale Complexity Measure for Deep Learning Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2401.09184 |