Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.01753 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909215849709568 |
|---|---|
| author | Lu, Fred Curtin, Ryan R. Raff, Edward Ferraro, Francis Holt, James |
| author_facet | Lu, Fred Curtin, Ryan R. Raff, Edward Ferraro, Francis Holt, James |
| contents | While distributed training is often viewed as a solution to optimizing linear models on increasingly large datasets, inter-machine communication costs of popular distributed approaches can dominate as data dimensionality increases. Recent work on non-interactive algorithms shows that approximate solutions for linear models can be obtained efficiently with only a single round of communication among machines. However, this approximation often degenerates as the number of machines increases. In this paper, building on the recent optimal weighted average method, we introduce a new technique, ACOWA, that allows an extra round of communication to achieve noticeably better approximation quality with minor runtime increases. Results show that for sparse distributed logistic regression, ACOWA obtains solutions that are more faithful to the empirical risk minimizer and attain substantially higher accuracy than other distributed algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_01753 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Optimizing the Optimal Weighted Average: Efficient Distributed Sparse Classification Lu, Fred Curtin, Ryan R. Raff, Edward Ferraro, Francis Holt, James Machine Learning Distributed, Parallel, and Cluster Computing While distributed training is often viewed as a solution to optimizing linear models on increasingly large datasets, inter-machine communication costs of popular distributed approaches can dominate as data dimensionality increases. Recent work on non-interactive algorithms shows that approximate solutions for linear models can be obtained efficiently with only a single round of communication among machines. However, this approximation often degenerates as the number of machines increases. In this paper, building on the recent optimal weighted average method, we introduce a new technique, ACOWA, that allows an extra round of communication to achieve noticeably better approximation quality with minor runtime increases. Results show that for sparse distributed logistic regression, ACOWA obtains solutions that are more faithful to the empirical risk minimizer and attain substantially higher accuracy than other distributed algorithms. |
| title | Optimizing the Optimal Weighted Average: Efficient Distributed Sparse Classification |
| topic | Machine Learning Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2406.01753 |