On the Burstiness of Distributed Machine Learning Traffic
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913181697310720 |
|---|---|
| author | Luangsomboon, Natchanon Fazel, Fahimeh Liebeherr, Jörg Sobhani, Ashkan Guan, Shichao Chu, Xingjun |
| author_facet | Luangsomboon, Natchanon Fazel, Fahimeh Liebeherr, Jörg Sobhani, Ashkan Guan, Shichao Chu, Xingjun |
| contents | Traffic from distributed training of machine learning (ML) models makes up a large and growing fraction of the traffic mix in enterprise data centers. While work on distributed ML abounds, the network traffic generated by distributed ML has received little attention. Using measurements on a testbed network, we investigate the traffic characteristics generated by the training of the ResNet-50 neural network with an emphasis on studying its short-term burstiness. For the latter we propose metrics that quantify traffic burstiness at different time scales. Our analysis reveals that distributed ML traffic exhibits a very high degree of burstiness on short time scales, exceeding a 60:1 peak-to-mean ratio on time intervals as long as 5~ms. We observe that training software orchestrates transmissions in such a way that burst transmissions from different sources within the same application do not result in congestion and packet losses. An extrapolation of the measurement data to multiple applications underscores the challenges of distributed ML traffic for congestion and flow control algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_00329 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | On the Burstiness of Distributed Machine Learning Traffic Luangsomboon, Natchanon Fazel, Fahimeh Liebeherr, Jörg Sobhani, Ashkan Guan, Shichao Chu, Xingjun Machine Learning Networking and Internet Architecture C.2.0; C.4 Traffic from distributed training of machine learning (ML) models makes up a large and growing fraction of the traffic mix in enterprise data centers. While work on distributed ML abounds, the network traffic generated by distributed ML has received little attention. Using measurements on a testbed network, we investigate the traffic characteristics generated by the training of the ResNet-50 neural network with an emphasis on studying its short-term burstiness. For the latter we propose metrics that quantify traffic burstiness at different time scales. Our analysis reveals that distributed ML traffic exhibits a very high degree of burstiness on short time scales, exceeding a 60:1 peak-to-mean ratio on time intervals as long as 5~ms. We observe that training software orchestrates transmissions in such a way that burst transmissions from different sources within the same application do not result in congestion and packet losses. An extrapolation of the measurement data to multiple applications underscores the challenges of distributed ML traffic for congestion and flow control algorithms. |
| title | On the Burstiness of Distributed Machine Learning Traffic |
| topic | Machine Learning Networking and Internet Architecture C.2.0; C.4 |
| url | https://arxiv.org/abs/2401.00329 |