Understanding Transfer Learning via Mean-field Analysis
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917813017378816 |
|---|---|
| author | Aminian, Gholamali Szpruch, Łukasz Cohen, Samuel N. |
| author_facet | Aminian, Gholamali Szpruch, Łukasz Cohen, Samuel N. |
| contents | We propose a novel framework for exploring generalization errors of transfer learning through the lens of differential calculus on the space of probability measures. In particular, we consider two main transfer learning scenarios, $α$-ERM and fine-tuning with the KL-regularized empirical risk minimization and establish generic conditions under which the generalization error and the population risk convergence rates for these scenarios are studied. Based on our theoretical results, we show the benefits of transfer learning with a one-hidden-layer neural network in the mean-field regime under some suitable integrability and regularity assumptions on the loss and activation functions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_17128 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Understanding Transfer Learning via Mean-field Analysis Aminian, Gholamali Szpruch, Łukasz Cohen, Samuel N. Machine Learning Functional Analysis We propose a novel framework for exploring generalization errors of transfer learning through the lens of differential calculus on the space of probability measures. In particular, we consider two main transfer learning scenarios, $α$-ERM and fine-tuning with the KL-regularized empirical risk minimization and establish generic conditions under which the generalization error and the population risk convergence rates for these scenarios are studied. Based on our theoretical results, we show the benefits of transfer learning with a one-hidden-layer neural network in the mean-field regime under some suitable integrability and regularity assumptions on the loss and activation functions. |
| title | Understanding Transfer Learning via Mean-field Analysis |
| topic | Machine Learning Functional Analysis |
| url | https://arxiv.org/abs/2410.17128 |