A PAC-Bayesian Link Between Generalisation and Flat Minima
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910820643897344 |
|---|---|
| author | Haddouche, Maxime Viallard, Paul Simsekli, Umut Guedj, Benjamin |
| author_facet | Haddouche, Maxime Viallard, Paul Simsekli, Umut Guedj, Benjamin |
| contents | Modern machine learning usually involves predictors in the overparameterised setting (number of trained parameters greater than dataset size), and their training yields not only good performance on training data, but also good generalisation capacity. This phenomenon challenges many theoretical results, and remains an open problem. To reach a better understanding, we provide novel generalisation bounds involving gradient terms. To do so, we combine the PAC-Bayes toolbox with Poincaré and Log-Sobolev inequalities, avoiding an explicit dependency on the dimension of the predictor space. Our results highlight the positive influence of flat minima (being minima with a neighbourhood nearly minimising the learning problem as well) on generalisation performance, involving directly the benefits of the optimisation phase. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_08508 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A PAC-Bayesian Link Between Generalisation and Flat Minima Haddouche, Maxime Viallard, Paul Simsekli, Umut Guedj, Benjamin Machine Learning Modern machine learning usually involves predictors in the overparameterised setting (number of trained parameters greater than dataset size), and their training yields not only good performance on training data, but also good generalisation capacity. This phenomenon challenges many theoretical results, and remains an open problem. To reach a better understanding, we provide novel generalisation bounds involving gradient terms. To do so, we combine the PAC-Bayes toolbox with Poincaré and Log-Sobolev inequalities, avoiding an explicit dependency on the dimension of the predictor space. Our results highlight the positive influence of flat minima (being minima with a neighbourhood nearly minimising the learning problem as well) on generalisation performance, involving directly the benefits of the optimisation phase. |
| title | A PAC-Bayesian Link Between Generalisation and Flat Minima |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.08508 |