A High-Dimensional Statistical Method for Optimizing Transfer Quantities in Multi-Source Transfer Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909872043327488 |
|---|---|
| author | Zhang, Qingyue Fu, Haohao Huang, Guanbo Liang, Yaoyuan Chu, Chang Peng, Tianren Wu, Yanru Li, Qi Li, Yang Huang, Shao-Lun |
| author_facet | Zhang, Qingyue Fu, Haohao Huang, Guanbo Liang, Yaoyuan Chu, Chang Peng, Tianren Wu, Yanru Li, Qi Li, Yang Huang, Shao-Lun |
| contents | Multi-source transfer learning provides an effective solution to data scarcity in real-world supervised learning scenarios by leveraging multiple source tasks. In this field, existing works typically use all available samples from sources in training, which constrains their training efficiency and may lead to suboptimal results. To address this, we propose a theoretical framework that answers the question: what is the optimal quantity of source samples needed from each source task to jointly train the target model? Specifically, we introduce a generalization error measure based on K-L divergence, and minimize it based on high-dimensional statistical analysis to determine the optimal transfer quantity for each source task. Additionally, we develop an architecture-agnostic and data-efficient algorithm OTQMS to implement our theoretical results for target model training in multi-source transfer learning. Experimental studies on diverse architectures and two real-world benchmark datasets show that our proposed algorithm significantly outperforms state-of-the-art approaches in both accuracy and data efficiency. The code and supplementary materials are available in https://github.com/zqy0126/OTQMS. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_04242 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A High-Dimensional Statistical Method for Optimizing Transfer Quantities in Multi-Source Transfer Learning Zhang, Qingyue Fu, Haohao Huang, Guanbo Liang, Yaoyuan Chu, Chang Peng, Tianren Wu, Yanru Li, Qi Li, Yang Huang, Shao-Lun Machine Learning Artificial Intelligence Multi-source transfer learning provides an effective solution to data scarcity in real-world supervised learning scenarios by leveraging multiple source tasks. In this field, existing works typically use all available samples from sources in training, which constrains their training efficiency and may lead to suboptimal results. To address this, we propose a theoretical framework that answers the question: what is the optimal quantity of source samples needed from each source task to jointly train the target model? Specifically, we introduce a generalization error measure based on K-L divergence, and minimize it based on high-dimensional statistical analysis to determine the optimal transfer quantity for each source task. Additionally, we develop an architecture-agnostic and data-efficient algorithm OTQMS to implement our theoretical results for target model training in multi-source transfer learning. Experimental studies on diverse architectures and two real-world benchmark datasets show that our proposed algorithm significantly outperforms state-of-the-art approaches in both accuracy and data efficiency. The code and supplementary materials are available in https://github.com/zqy0126/OTQMS. |
| title | A High-Dimensional Statistical Method for Optimizing Transfer Quantities in Multi-Source Transfer Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2502.04242 |