Scalable Nested Optimization for Deep Learning
Fuente:
arXiv
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914854916325376 |
|---|---|
| author | Lorraine, Jonathan |
| author_facet | Lorraine, Jonathan |
| contents | Gradient-based optimization has been critical to the success of machine learning, updating a single set of parameters to minimize a single loss. A growing number of applications rely on a generalization of this, where we have a bilevel or nested optimization of which subsets of parameters update on different objectives nested inside each other. We focus on motivating examples of hyperparameter optimization and generative adversarial networks. However, naively applying classical methods often fails when we look at solving these nested problems on a large scale. In this thesis, we build tools for nested optimization that scale to deep learning setups. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_01526 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Scalable Nested Optimization for Deep Learning Lorraine, Jonathan Machine Learning Artificial Intelligence Neural and Evolutionary Computing Optimization and Control 68T05 I.2.6; I.2.8; I.5.1; G.1.6 Gradient-based optimization has been critical to the success of machine learning, updating a single set of parameters to minimize a single loss. A growing number of applications rely on a generalization of this, where we have a bilevel or nested optimization of which subsets of parameters update on different objectives nested inside each other. We focus on motivating examples of hyperparameter optimization and generative adversarial networks. However, naively applying classical methods often fails when we look at solving these nested problems on a large scale. In this thesis, we build tools for nested optimization that scale to deep learning setups. |
| title | Scalable Nested Optimization for Deep Learning |
| topic | Machine Learning Artificial Intelligence Neural and Evolutionary Computing Optimization and Control 68T05 I.2.6; I.2.8; I.5.1; G.1.6 |
| url | https://arxiv.org/abs/2407.01526 |