cMPI: Using CXL Memory Sharing for MPI One-Sided and Two-Sided Inter-Node Communications
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911316393852928 |
|---|---|
| author | Wang, Xi Ma, Bin Kim, Jongryool Koh, Byungil Kim, Hoshik Li, Dong |
| author_facet | Wang, Xi Ma, Bin Kim, Jongryool Koh, Byungil Kim, Hoshik Li, Dong |
| contents | Message Passing Interface (MPI) is a foundational programming model for high-performance computing. MPI libraries traditionally employ network interconnects (e.g., Ethernet and InfiniBand) and network protocols (e.g., TCP and RoCE) with complex software stacks for cross-node communication. We present cMPI, the first work to optimize MPI point-to-point communication (both one-sided and two-sided) using CXL memory sharing on a real CXL platform, transforming cross-node communication into memory transactions and data copies within CXL memory, bypassing traditional network protocols. We analyze performance across various interconnects and find that CXL memory sharing achieves 7.2x-8.1x lower latency than TCP-based interconnects deployed in small- and medium-scale clusters. We address challenges of CXL memory sharing for MPI communication, including data object management over the dax representation [50], cache coherence, and atomic operations. Overall, cMPI outperforms TCP over standard Ethernet NIC and high-end SmartNIC by up to 49x and 72x in latency and bandwidth, respectively, for small messages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_05476 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | cMPI: Using CXL Memory Sharing for MPI One-Sided and Two-Sided Inter-Node Communications Wang, Xi Ma, Bin Kim, Jongryool Koh, Byungil Kim, Hoshik Li, Dong Distributed, Parallel, and Cluster Computing Hardware Architecture Networking and Internet Architecture Message Passing Interface (MPI) is a foundational programming model for high-performance computing. MPI libraries traditionally employ network interconnects (e.g., Ethernet and InfiniBand) and network protocols (e.g., TCP and RoCE) with complex software stacks for cross-node communication. We present cMPI, the first work to optimize MPI point-to-point communication (both one-sided and two-sided) using CXL memory sharing on a real CXL platform, transforming cross-node communication into memory transactions and data copies within CXL memory, bypassing traditional network protocols. We analyze performance across various interconnects and find that CXL memory sharing achieves 7.2x-8.1x lower latency than TCP-based interconnects deployed in small- and medium-scale clusters. We address challenges of CXL memory sharing for MPI communication, including data object management over the dax representation [50], cache coherence, and atomic operations. Overall, cMPI outperforms TCP over standard Ethernet NIC and high-end SmartNIC by up to 49x and 72x in latency and bandwidth, respectively, for small messages. |
| title | cMPI: Using CXL Memory Sharing for MPI One-Sided and Two-Sided Inter-Node Communications |
| topic | Distributed, Parallel, and Cluster Computing Hardware Architecture Networking and Internet Architecture |
| url | https://arxiv.org/abs/2510.05476 |