On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility View
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915456956235776 |
|---|---|
| author | Guo, Tao Wang, Junxiao Huo, Fushuo Cui, Laizhong Guo, Song Gui, Jie Tao, Dacheng |
| author_facet | Guo, Tao Wang, Junxiao Huo, Fushuo Cui, Laizhong Guo, Song Gui, Jie Tao, Dacheng |
| contents | Federated Learning (FL) enables training models across decentralized data silos while preserving client data privacy. Recent research has explored efficient methods for post-training large language models (LLMs) within FL to address computational and communication challenges. While existing approaches often rely on access to LLMs' internal information, which is frequently restricted in real-world scenarios, an inference-only paradigm (black-box FedLLM) has emerged to address these limitations. This paper presents a comprehensive survey on federated tuning for LLMs. We propose a taxonomy categorizing existing studies along two axes: model access-based and parameter efficiency-based optimization. We classify FedLLM approaches into white-box, gray-box, and black-box techniques, highlighting representative methods within each category. We review emerging research treating LLMs as black-box inference APIs and discuss promising directions and open challenges for future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_16261 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility View Guo, Tao Wang, Junxiao Huo, Fushuo Cui, Laizhong Guo, Song Gui, Jie Tao, Dacheng Machine Learning Federated Learning (FL) enables training models across decentralized data silos while preserving client data privacy. Recent research has explored efficient methods for post-training large language models (LLMs) within FL to address computational and communication challenges. While existing approaches often rely on access to LLMs' internal information, which is frequently restricted in real-world scenarios, an inference-only paradigm (black-box FedLLM) has emerged to address these limitations. This paper presents a comprehensive survey on federated tuning for LLMs. We propose a taxonomy categorizing existing studies along two axes: model access-based and parameter efficiency-based optimization. We classify FedLLM approaches into white-box, gray-box, and black-box techniques, highlighting representative methods within each category. We review emerging research treating LLMs as black-box inference APIs and discuss promising directions and open challenges for future research. |
| title | On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility View |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2508.16261 |