Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929368311267328 |
|---|---|
| author | Fuhrer, Benjamin Shpigelman, Yuval Tessler, Chen Mannor, Shie Chechik, Gal Zahavi, Eitan Dalal, Gal |
| author_facet | Fuhrer, Benjamin Shpigelman, Yuval Tessler, Chen Mannor, Shie Chechik, Gal Zahavi, Eitan Dalal, Gal |
| contents | As communication protocols evolve, datacenter network utilization increases. As a result, congestion is more frequent, causing higher latency and packet loss. Combined with the increasing complexity of workloads, manual design of congestion control (CC) algorithms becomes extremely difficult. This calls for the development of AI approaches to replace the human effort. Unfortunately, it is currently not possible to deploy AI models on network devices due to their limited computational capabilities. Here, we offer a solution to this problem by building a computationally-light solution based on a recent reinforcement learning CC algorithm [arXiv:2207.02295]. We reduce the inference time of RL-CC by x500 by distilling its complex neural network into decision trees. This transformation enables real-time inference within the $μ$-sec decision-time requirement, with a negligible effect on quality. We deploy the transformed policy on NVIDIA NICs in a live cluster. Compared to popular CC algorithms used in production, RL-CC is the only method that performs well on all benchmarks tested over a large range of number of flows. It balances multiple metrics simultaneously: bandwidth, latency, and packet drops. These results suggest that data-driven methods for CC are feasible, challenging the prior belief that handcrafted heuristics are necessary to achieve optimal performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2207_02295 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs Fuhrer, Benjamin Shpigelman, Yuval Tessler, Chen Mannor, Shie Chechik, Gal Zahavi, Eitan Dalal, Gal Networking and Internet Architecture Artificial Intelligence Machine Learning As communication protocols evolve, datacenter network utilization increases. As a result, congestion is more frequent, causing higher latency and packet loss. Combined with the increasing complexity of workloads, manual design of congestion control (CC) algorithms becomes extremely difficult. This calls for the development of AI approaches to replace the human effort. Unfortunately, it is currently not possible to deploy AI models on network devices due to their limited computational capabilities. Here, we offer a solution to this problem by building a computationally-light solution based on a recent reinforcement learning CC algorithm [arXiv:2207.02295]. We reduce the inference time of RL-CC by x500 by distilling its complex neural network into decision trees. This transformation enables real-time inference within the $μ$-sec decision-time requirement, with a negligible effect on quality. We deploy the transformed policy on NVIDIA NICs in a live cluster. Compared to popular CC algorithms used in production, RL-CC is the only method that performs well on all benchmarks tested over a large range of number of flows. It balances multiple metrics simultaneously: bandwidth, latency, and packet drops. These results suggest that data-driven methods for CC are feasible, challenging the prior belief that handcrafted heuristics are necessary to achieve optimal performance. |
| title | Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs |
| topic | Networking and Internet Architecture Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2207.02295 |