Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.23593 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917164761481216 |
|---|---|
| author | Agrawal, Nikita Mertel, Simon Mayer, Ruben |
| author_facet | Agrawal, Nikita Mertel, Simon Mayer, Ruben |
| contents | Post-training of foundation language models has emerged as a promising research domain in federated learning (FL) with the goal to enable privacy-preserving model improvements and adaptations to user's downstream tasks. Recent advances in this area adopt centralized post-training approaches that build upon black-box foundation language models where there is no access to model weights and architecture details. Although the use of black-box models has been successful in centralized post-training, their blind replication in FL raises several concerns. Our position is that using black-box models in FL contradicts the core principles of federation such as data privacy and autonomy. In this position paper, we critically analyze the usage of black-box models in federated post-training, and provide a detailed account of various aspects of openness and their implications for FL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_23593 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Position: Federated Foundation Language Model Post-Training Should Focus on Open-Source Models Agrawal, Nikita Mertel, Simon Mayer, Ruben Machine Learning Post-training of foundation language models has emerged as a promising research domain in federated learning (FL) with the goal to enable privacy-preserving model improvements and adaptations to user's downstream tasks. Recent advances in this area adopt centralized post-training approaches that build upon black-box foundation language models where there is no access to model weights and architecture details. Although the use of black-box models has been successful in centralized post-training, their blind replication in FL raises several concerns. Our position is that using black-box models in FL contradicts the core principles of federation such as data privacy and autonomy. In this position paper, we critically analyze the usage of black-box models in federated post-training, and provide a detailed account of various aspects of openness and their implications for FL. |
| title | Position: Federated Foundation Language Model Post-Training Should Focus on Open-Source Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.23593 |