Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.00620 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916770575548416 |
|---|---|
| author | Chung, Ming-Yu Fan, Jiashuo Ye, Hancheng Wang, Qinsi Shen, Wei-Chen Yu, Chia-Mu Chen, Pin-Yu Kuo, Sy-Yen |
| author_facet | Chung, Ming-Yu Fan, Jiashuo Ye, Hancheng Wang, Qinsi Shen, Wei-Chen Yu, Chia-Mu Chen, Pin-Yu Kuo, Sy-Yen |
| contents | Model Reprogramming (MR) is a resource-efficient framework that adapts large pre-trained models to new tasks with minimal additional parameters and data, offering a promising solution to the challenges of training large models for diverse tasks. Despite its empirical success across various domains such as computer vision and time-series forecasting, the theoretical foundations of MR remain underexplored. In this paper, we present a comprehensive theoretical analysis of MR through the lens of the Neural Tangent Kernel (NTK) framework. We demonstrate that the success of MR is governed by the eigenvalue spectrum of the NTK matrix on the target dataset and establish the critical role of the source model's effectiveness in determining reprogramming outcomes. Our contributions include a novel theoretical framework for MR, insights into the relationship between source and target models, and extensive experiments validating our findings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_00620 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Model Reprogramming Demystified: A Neural Tangent Kernel Perspective Chung, Ming-Yu Fan, Jiashuo Ye, Hancheng Wang, Qinsi Shen, Wei-Chen Yu, Chia-Mu Chen, Pin-Yu Kuo, Sy-Yen Machine Learning 68T05, 46E22 Model Reprogramming (MR) is a resource-efficient framework that adapts large pre-trained models to new tasks with minimal additional parameters and data, offering a promising solution to the challenges of training large models for diverse tasks. Despite its empirical success across various domains such as computer vision and time-series forecasting, the theoretical foundations of MR remain underexplored. In this paper, we present a comprehensive theoretical analysis of MR through the lens of the Neural Tangent Kernel (NTK) framework. We demonstrate that the success of MR is governed by the eigenvalue spectrum of the NTK matrix on the target dataset and establish the critical role of the source model's effectiveness in determining reprogramming outcomes. Our contributions include a novel theoretical framework for MR, insights into the relationship between source and target models, and extensive experiments validating our findings. |
| title | Model Reprogramming Demystified: A Neural Tangent Kernel Perspective |
| topic | Machine Learning 68T05, 46E22 |
| url | https://arxiv.org/abs/2506.00620 |