On the inductive bias of infinite-depth ResNets and the bottleneck rank
Fuente:
arXiv
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915131226587136 |
|---|---|
| author | Boix-Adsera, Enric |
| author_facet | Boix-Adsera, Enric |
| contents | We compute the minimum-norm weights of a deep linear ResNet, and find that the inductive bias of this architecture lies between minimizing nuclear norm and rank. This implies that, with appropriate hyperparameters, deep nonlinear ResNets have an inductive bias towards minimizing bottleneck rank. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_19149 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the inductive bias of infinite-depth ResNets and the bottleneck rank Boix-Adsera, Enric Machine Learning Artificial Intelligence We compute the minimum-norm weights of a deep linear ResNet, and find that the inductive bias of this architecture lies between minimizing nuclear norm and rank. This implies that, with appropriate hyperparameters, deep nonlinear ResNets have an inductive bias towards minimizing bottleneck rank. |
| title | On the inductive bias of infinite-depth ResNets and the bottleneck rank |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2501.19149 |