Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglement
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916593169072128 |
|---|---|
| author | Patel, Gaurav Sandino, Christopher Mahasseni, Behrooz Zippi, Ellen L Azemi, Erdrin Moin, Ali Minxha, Juri |
| author_facet | Patel, Gaurav Sandino, Christopher Mahasseni, Behrooz Zippi, Ellen L Azemi, Erdrin Moin, Ali Minxha, Juri |
| contents | In this paper, we propose a framework for efficient Source-Free Domain Adaptation (SFDA) in the context of time-series, focusing on enhancing both parameter efficiency and data-sample utilization. Our approach introduces an improved paradigm for source-model preparation and target-side adaptation, aiming to enhance training efficiency during target adaptation. Specifically, we reparameterize the source model's weights in a Tucker-style decomposed manner, factorizing the model into a compact form during the source model preparation phase. During target-side adaptation, only a subset of these decomposed factors is fine-tuned, leading to significant improvements in training efficiency. We demonstrate using PAC Bayesian analysis that this selective fine-tuning strategy implicitly regularizes the adaptation process by constraining the model's learning capacity. Furthermore, this re-parameterization reduces the overall model size and enhances inference efficiency, making the approach particularly well suited for resource-constrained devices. Additionally, we demonstrate that our framework is compatible with various SFDA methods and achieves significant computational efficiency, reducing the number of fine-tuned parameters and inference overhead in terms of MACs by over 90% while maintaining model performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_02147 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglement Patel, Gaurav Sandino, Christopher Mahasseni, Behrooz Zippi, Ellen L Azemi, Erdrin Moin, Ali Minxha, Juri Machine Learning Artificial Intelligence Signal Processing In this paper, we propose a framework for efficient Source-Free Domain Adaptation (SFDA) in the context of time-series, focusing on enhancing both parameter efficiency and data-sample utilization. Our approach introduces an improved paradigm for source-model preparation and target-side adaptation, aiming to enhance training efficiency during target adaptation. Specifically, we reparameterize the source model's weights in a Tucker-style decomposed manner, factorizing the model into a compact form during the source model preparation phase. During target-side adaptation, only a subset of these decomposed factors is fine-tuned, leading to significant improvements in training efficiency. We demonstrate using PAC Bayesian analysis that this selective fine-tuning strategy implicitly regularizes the adaptation process by constraining the model's learning capacity. Furthermore, this re-parameterization reduces the overall model size and enhances inference efficiency, making the approach particularly well suited for resource-constrained devices. Additionally, we demonstrate that our framework is compatible with various SFDA methods and achieves significant computational efficiency, reducing the number of fine-tuned parameters and inference overhead in terms of MACs by over 90% while maintaining model performance. |
| title | Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglement |
| topic | Machine Learning Artificial Intelligence Signal Processing |
| url | https://arxiv.org/abs/2410.02147 |