SDR-RDMA: Software-Defined Reliability Architecture for Planetary Scale RDMA Communication
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913831270219776 |
|---|---|
| author | Khalilov, Mikhail Shen, Siyuan Chrapek, Marcin Chen, Tiancheng Nakano, Kenji Gootzen, Peter-Jan Di Girolamo, Salvatore Nudelman, Rami Bloch, Gil Anantharamu, Sreevatsa Elhaddad, Mahmoud Jose, Jithin Kabbani, Abdul Moe, Scott Taranov, Konstantin Yu, Zhuolong Zhang, Jie Mazzoletti, Nicola Hoefler, Torsten |
| author_facet | Khalilov, Mikhail Shen, Siyuan Chrapek, Marcin Chen, Tiancheng Nakano, Kenji Gootzen, Peter-Jan Di Girolamo, Salvatore Nudelman, Rami Bloch, Gil Anantharamu, Sreevatsa Elhaddad, Mahmoud Jose, Jithin Kabbani, Abdul Moe, Scott Taranov, Konstantin Yu, Zhuolong Zhang, Jie Mazzoletti, Nicola Hoefler, Torsten |
| contents | RDMA is vital for efficient distributed training across datacenters, but millisecond-scale latencies complicate the design of its reliability layer. We show that depending on long-haul link characteristics, such as drop rate, distance and bandwidth, the widely used Selective Repeat algorithm can be inefficient, warranting alternatives like Erasure Coding. To enable such alternatives on existing hardware, we propose SDR-RDMA, a software-defined reliability stack for RDMA. Its core is a lightweight SDR SDK that extends standard point-to-point RDMA semantics -- fundamental to AI networking stacks -- with a receive buffer bitmap. SDR bitmap enables partial message completion to let applications implement custom reliability schemes tailored to specific deployments, while preserving zero-copy RDMA benefits. By offloading the SDR backend to NVIDIA's Data Path Accelerator (DPA), we achieve line-rate performance, enabling efficient inter-datacenter communication and advancing reliability innovation for inter-datacenter training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_05366 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SDR-RDMA: Software-Defined Reliability Architecture for Planetary Scale RDMA Communication Khalilov, Mikhail Shen, Siyuan Chrapek, Marcin Chen, Tiancheng Nakano, Kenji Gootzen, Peter-Jan Di Girolamo, Salvatore Nudelman, Rami Bloch, Gil Anantharamu, Sreevatsa Elhaddad, Mahmoud Jose, Jithin Kabbani, Abdul Moe, Scott Taranov, Konstantin Yu, Zhuolong Zhang, Jie Mazzoletti, Nicola Hoefler, Torsten Networking and Internet Architecture RDMA is vital for efficient distributed training across datacenters, but millisecond-scale latencies complicate the design of its reliability layer. We show that depending on long-haul link characteristics, such as drop rate, distance and bandwidth, the widely used Selective Repeat algorithm can be inefficient, warranting alternatives like Erasure Coding. To enable such alternatives on existing hardware, we propose SDR-RDMA, a software-defined reliability stack for RDMA. Its core is a lightweight SDR SDK that extends standard point-to-point RDMA semantics -- fundamental to AI networking stacks -- with a receive buffer bitmap. SDR bitmap enables partial message completion to let applications implement custom reliability schemes tailored to specific deployments, while preserving zero-copy RDMA benefits. By offloading the SDR backend to NVIDIA's Data Path Accelerator (DPA), we achieve line-rate performance, enabling efficient inter-datacenter communication and advancing reliability innovation for inter-datacenter training. |
| title | SDR-RDMA: Software-Defined Reliability Architecture for Planetary Scale RDMA Communication |
| topic | Networking and Internet Architecture |
| url | https://arxiv.org/abs/2505.05366 |