Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911230422155264 |
|---|---|
| author | Cheng, Ziheng Xie, Tianyu Zhang, Shiyue Zhang, Cheng |
| author_facet | Cheng, Ziheng Xie, Tianyu Zhang, Shiyue Zhang, Cheng |
| contents | While conditional diffusion models have achieved remarkable success in various applications, they require abundant data to train from scratch, which is often infeasible in practice. To address this issue, transfer learning has emerged as an essential paradigm in small data regimes. Despite its empirical success, the theoretical underpinnings of transfer learning conditional diffusion models remain unexplored. In this paper, we take the first step towards understanding the sample efficiency of transfer learning conditional diffusion models through the lens of representation learning. Inspired by practical training procedures, we assume that there exists a low-dimensional representation of conditions shared across all tasks. Our analysis shows that with a well-learned representation from source tasks, the samplecomplexity of target tasks can be reduced substantially. In addition, we investigate the practical implications of our theoretical results in several real-world applications of conditional diffusion models. Numerical experiments are also conducted to verify our results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_04491 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learning Cheng, Ziheng Xie, Tianyu Zhang, Shiyue Zhang, Cheng Machine Learning Statistics Theory While conditional diffusion models have achieved remarkable success in various applications, they require abundant data to train from scratch, which is often infeasible in practice. To address this issue, transfer learning has emerged as an essential paradigm in small data regimes. Despite its empirical success, the theoretical underpinnings of transfer learning conditional diffusion models remain unexplored. In this paper, we take the first step towards understanding the sample efficiency of transfer learning conditional diffusion models through the lens of representation learning. Inspired by practical training procedures, we assume that there exists a low-dimensional representation of conditions shared across all tasks. Our analysis shows that with a well-learned representation from source tasks, the samplecomplexity of target tasks can be reduced substantially. In addition, we investigate the practical implications of our theoretical results in several real-world applications of conditional diffusion models. Numerical experiments are also conducted to verify our results. |
| title | Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learning |
| topic | Machine Learning Statistics Theory |
| url | https://arxiv.org/abs/2502.04491 |