On Robust Aggregation for Distributed Data
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909511870054400 |
|---|---|
| author | Li, Xian Liang, Xuan Welsh, A. H. Zou, Tao |
| author_facet | Li, Xian Liang, Xuan Welsh, A. H. Zou, Tao |
| contents | When data are stored across multiple locations, directly pooling all the data together for statistical analysis may be impossible due to communication costs and privacy concerns. Distributed computing systems allow the analysis of such data, by getting local servers to separately process their own statistical analyses and using a central processor to aggregate the local statistical results. Naive aggregation of local statistics using simple or weighted averages, is vulnerable to contamination within a distributed computing system. This paper develops and investigates a Huber-type aggregation method for locally computed M-estimators to handle contamination in the local estimates. Our implementation of this aggregation method requires estimating the asymptotic variance-covariance matrix of the M-estimator, which we accomplish using a robust spatial median approach. Theoretically, the Huber-type aggregation achieves the same convergence rate as if all the data were pooled. We establish its asymptotic normality for making inferences, including justifying a two-step approach for detecting contamination in the distributed computing system. Extensive simulation studies are conducted to validate the theoretical results and the usefulness of our proposed approach is demonstrated on U.S. airline data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_18740 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On Robust Aggregation for Distributed Data Li, Xian Liang, Xuan Welsh, A. H. Zou, Tao Methodology 62F35, 62F12, 68W15 When data are stored across multiple locations, directly pooling all the data together for statistical analysis may be impossible due to communication costs and privacy concerns. Distributed computing systems allow the analysis of such data, by getting local servers to separately process their own statistical analyses and using a central processor to aggregate the local statistical results. Naive aggregation of local statistics using simple or weighted averages, is vulnerable to contamination within a distributed computing system. This paper develops and investigates a Huber-type aggregation method for locally computed M-estimators to handle contamination in the local estimates. Our implementation of this aggregation method requires estimating the asymptotic variance-covariance matrix of the M-estimator, which we accomplish using a robust spatial median approach. Theoretically, the Huber-type aggregation achieves the same convergence rate as if all the data were pooled. We establish its asymptotic normality for making inferences, including justifying a two-step approach for detecting contamination in the distributed computing system. Extensive simulation studies are conducted to validate the theoretical results and the usefulness of our proposed approach is demonstrated on U.S. airline data. |
| title | On Robust Aggregation for Distributed Data |
| topic | Methodology 62F35, 62F12, 68W15 |
| url | https://arxiv.org/abs/2502.18740 |