Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2505.23024 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914043762049024 |
|---|---|
| author | Wang, Zhihao Huang, Wenke Chen, Tian Shi, Zekun Wan, Guancheng Qiao, Yu Yang, Bin Wang, Jian Li, Bing Ye, Mang |
| author_facet | Wang, Zhihao Huang, Wenke Chen, Tian Shi, Zekun Wan, Guancheng Qiao, Yu Yang, Bin Wang, Jian Li, Bing Ye, Mang |
| contents | The Vision Language Model (VLM) excels in aligning vision and language representations, and prompt learning has emerged as a key technique for adapting such models to downstream tasks. However, the application of prompt learning with VLM in federated learning (FL) scenarios remains underexplored. This paper systematically investigates the behavioral differences between language prompt learning (LPT) and vision prompt learning (VPT) under data heterogeneity challenges, including label skew and domain shift. We conduct extensive experiments to evaluate the impact of various FL and prompt configurations, such as client scale, aggregation strategies, and prompt length, to assess the robustness of Federated Prompt Learning (FPL). Furthermore, we explore strategies for enhancing prompt learning in complex scenarios where label skew and domain shift coexist, including leveraging both prompt types when computational resources allow. Our findings offer practical insights into optimizing prompt learning in federated settings, contributing to the broader deployment of VLMs in privacy-preserving environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_23024 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An Empirical Study of Federated Prompt Learning for Vision Language Model Wang, Zhihao Huang, Wenke Chen, Tian Shi, Zekun Wan, Guancheng Qiao, Yu Yang, Bin Wang, Jian Li, Bing Ye, Mang Machine Learning The Vision Language Model (VLM) excels in aligning vision and language representations, and prompt learning has emerged as a key technique for adapting such models to downstream tasks. However, the application of prompt learning with VLM in federated learning (FL) scenarios remains underexplored. This paper systematically investigates the behavioral differences between language prompt learning (LPT) and vision prompt learning (VPT) under data heterogeneity challenges, including label skew and domain shift. We conduct extensive experiments to evaluate the impact of various FL and prompt configurations, such as client scale, aggregation strategies, and prompt length, to assess the robustness of Federated Prompt Learning (FPL). Furthermore, we explore strategies for enhancing prompt learning in complex scenarios where label skew and domain shift coexist, including leveraging both prompt types when computational resources allow. Our findings offer practical insights into optimizing prompt learning in federated settings, contributing to the broader deployment of VLMs in privacy-preserving environments. |
| title | An Empirical Study of Federated Prompt Learning for Vision Language Model |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.23024 |