How to induce regularization in linear models: A guide to reparametrizing gradient flow
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913255630307328 |
|---|---|
| author | Chou, Hung-Hsu Maly, Johannes Stöger, Dominik |
| author_facet | Chou, Hung-Hsu Maly, Johannes Stöger, Dominik |
| contents | In this work, we analyze the relation between reparametrizations of gradient flow and the induced implicit bias in linear models, which encompass various basic regression tasks. In particular, we aim at understanding the influence of the model parameters - reparametrization, loss, and link function - on the convergence behavior of gradient flow. Our results provide conditions under which the implicit bias can be well-described and convergence of the flow is guaranteed. We furthermore show how to use these insights for designing reparametrization functions that lead to specific implicit biases which are closely connected to $\ell_p$- or trigonometric regularizers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_04921 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | How to induce regularization in linear models: A guide to reparametrizing gradient flow Chou, Hung-Hsu Maly, Johannes Stöger, Dominik Optimization and Control Numerical Analysis In this work, we analyze the relation between reparametrizations of gradient flow and the induced implicit bias in linear models, which encompass various basic regression tasks. In particular, we aim at understanding the influence of the model parameters - reparametrization, loss, and link function - on the convergence behavior of gradient flow. Our results provide conditions under which the implicit bias can be well-described and convergence of the flow is guaranteed. We furthermore show how to use these insights for designing reparametrization functions that lead to specific implicit biases which are closely connected to $\ell_p$- or trigonometric regularizers. |
| title | How to induce regularization in linear models: A guide to reparametrizing gradient flow |
| topic | Optimization and Control Numerical Analysis |
| url | https://arxiv.org/abs/2308.04921 |