Trustworthy Efficient Communication for Distributed Learning using LQ-SGD Algorithm
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913906714214400 |
|---|---|
| author | Li, Hongyang Bai, Lincen Wu, Caesar Chadli, Mohammed Mammar, Said Bouvry, Pascal |
| author_facet | Li, Hongyang Bai, Lincen Wu, Caesar Chadli, Mohammed Mammar, Said Bouvry, Pascal |
| contents | We propose LQ-SGD (Low-Rank Quantized Stochastic Gradient Descent), an efficient communication gradient compression algorithm designed for distributed training. LQ-SGD further develops on the basis of PowerSGD by incorporating the low-rank approximation and log-quantization techniques, which drastically reduce the communication overhead, while still ensuring the convergence speed of training and model accuracy. In addition, LQ-SGD and other compression-based methods show stronger resistance to gradient inversion than traditional SGD, providing a more robust and efficient optimization path for distributed learning systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_17974 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Trustworthy Efficient Communication for Distributed Learning using LQ-SGD Algorithm Li, Hongyang Bai, Lincen Wu, Caesar Chadli, Mohammed Mammar, Said Bouvry, Pascal Machine Learning We propose LQ-SGD (Low-Rank Quantized Stochastic Gradient Descent), an efficient communication gradient compression algorithm designed for distributed training. LQ-SGD further develops on the basis of PowerSGD by incorporating the low-rank approximation and log-quantization techniques, which drastically reduce the communication overhead, while still ensuring the convergence speed of training and model accuracy. In addition, LQ-SGD and other compression-based methods show stronger resistance to gradient inversion than traditional SGD, providing a more robust and efficient optimization path for distributed learning systems. |
| title | Trustworthy Efficient Communication for Distributed Learning using LQ-SGD Algorithm |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.17974 |