Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2509.15147 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918407924875264 |
|---|---|
| author | Kovalchuk, Viktor Son, Denis Bolatov, Arman Guizani, Mohsen Horváth, Samuel Panov, Maxim Takáč, Martin Gorbunov, Eduard Kotelevskii, Nikita |
| author_facet | Kovalchuk, Viktor Son, Denis Bolatov, Arman Guizani, Mohsen Horváth, Samuel Panov, Maxim Takáč, Martin Gorbunov, Eduard Kotelevskii, Nikita |
| contents | Under data heterogeneity (e.g., $\textit{class mismatch}$), clients may produce unreliable predictions for instances belonging to unfamiliar classes. An equally weighted combination of such predictions can corrupt the teacher signal used for distillation. In this paper, we provide a theoretical analysis of Federated Distillation and show that aggregating client predictions on a shared public dataset converges to a neighborhood of the optimum, where the neighborhood size is governed by the aggregation quality. We further propose two uncertainty-aware aggregation methods, $\mathbf{UWA}$ and $\mathbf{sUWA}$, which leverage density-based uncertainty estimates to down-weight unreliable client predictions. Experiments on image and text classification benchmarks demonstrate that our methods are particularly effective under high data heterogeneity, while matching standard averaging when heterogeneity is low. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_15147 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Who to Trust? Aggregating Client Predictions in Federated Distillation Kovalchuk, Viktor Son, Denis Bolatov, Arman Guizani, Mohsen Horváth, Samuel Panov, Maxim Takáč, Martin Gorbunov, Eduard Kotelevskii, Nikita Machine Learning Under data heterogeneity (e.g., $\textit{class mismatch}$), clients may produce unreliable predictions for instances belonging to unfamiliar classes. An equally weighted combination of such predictions can corrupt the teacher signal used for distillation. In this paper, we provide a theoretical analysis of Federated Distillation and show that aggregating client predictions on a shared public dataset converges to a neighborhood of the optimum, where the neighborhood size is governed by the aggregation quality. We further propose two uncertainty-aware aggregation methods, $\mathbf{UWA}$ and $\mathbf{sUWA}$, which leverage density-based uncertainty estimates to down-weight unreliable client predictions. Experiments on image and text classification benchmarks demonstrate that our methods are particularly effective under high data heterogeneity, while matching standard averaging when heterogeneity is low. |
| title | Who to Trust? Aggregating Client Predictions in Federated Distillation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2509.15147 |