Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2021
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2102.11210 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913923180003328 |
|---|---|
| author | Sandler, Adam Klabjan, Diego Luo, Yuan |
| author_facet | Sandler, Adam Klabjan, Diego Luo, Yuan |
| contents | We develop regularization methods to find flat minima while training deep neural networks. These minima generalize better than sharp minima, yielding models outperforming baselines on real-world test data (which may be distributed differently than the training data). Specifically, we propose a method of regularized optimization to reduce the spectral radius of the Hessian of the loss function. We also derive algorithms to efficiently optimize neural network models and prove that these algorithms almost surely converge. Furthermore, we demonstrate that our algorithm works effectively on applications in different domains, including healthcare. To show that our models generalize well, we introduced various methods for testing generalizability and found that our models outperform comparable baseline models on these tests. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2102_11210 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | Non-Convex Optimization with Spectral Radius Regularization Sandler, Adam Klabjan, Diego Luo, Yuan Machine Learning We develop regularization methods to find flat minima while training deep neural networks. These minima generalize better than sharp minima, yielding models outperforming baselines on real-world test data (which may be distributed differently than the training data). Specifically, we propose a method of regularized optimization to reduce the spectral radius of the Hessian of the loss function. We also derive algorithms to efficiently optimize neural network models and prove that these algorithms almost surely converge. Furthermore, we demonstrate that our algorithm works effectively on applications in different domains, including healthcare. To show that our models generalize well, we introduced various methods for testing generalizability and found that our models outperform comparable baseline models on these tests. |
| title | Non-Convex Optimization with Spectral Radius Regularization |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2102.11210 |