Co-Design and Evaluation of a CPU-Free MPI GPU Communication Abstraction and Implementation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910042120257536 |
|---|---|
| author | Bridges, Patrick G. Schafer, Derek Lange, Jack White III, James B. Skjellum, Anthony Suggs, Evan Hines, Thomas Bangalore, Purushotham Dosanjh, Matthew G. F. Schonbein, Whit |
| author_facet | Bridges, Patrick G. Schafer, Derek Lange, Jack White III, James B. Skjellum, Anthony Suggs, Evan Hines, Thomas Bangalore, Purushotham Dosanjh, Matthew G. F. Schonbein, Whit |
| contents | Removing the CPU from the communication fast path is essential to efficient GPU-based ML and HPC application performance. However, existing GPU communication APIs either continue to rely on the CPU for communication or rely on APIs that place significant synchronization burdens on programmers. In this paper we describe the design, implementation, and evaluation of an MPI-based GPU communication API enabling easy-to-use, high-performance, CPU-free communication. This API builds on previously proposed MPI extensions and leverages HPE Slingshot 11 network card capabilities. We demonstrate the utility and performance of the API by showing how the API naturally enables CPU-free gather/scatter halo exchange communication primitives in the Cabana/Kokkos performance portability framework, and through a performance comparison with Cray MPICH on the Frontier and Tuolumne supercomputers. Results from this evaluation show up to a 50% reduction in medium message latency in simple GPU ping-pong exchanges and a 28% speedup improvement when strong scaling a halo-exchange benchmark to 8,192 GPUs of the Frontier supercomputer. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15356 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Co-Design and Evaluation of a CPU-Free MPI GPU Communication Abstraction and Implementation Bridges, Patrick G. Schafer, Derek Lange, Jack White III, James B. Skjellum, Anthony Suggs, Evan Hines, Thomas Bangalore, Purushotham Dosanjh, Matthew G. F. Schonbein, Whit Distributed, Parallel, and Cluster Computing Removing the CPU from the communication fast path is essential to efficient GPU-based ML and HPC application performance. However, existing GPU communication APIs either continue to rely on the CPU for communication or rely on APIs that place significant synchronization burdens on programmers. In this paper we describe the design, implementation, and evaluation of an MPI-based GPU communication API enabling easy-to-use, high-performance, CPU-free communication. This API builds on previously proposed MPI extensions and leverages HPE Slingshot 11 network card capabilities. We demonstrate the utility and performance of the API by showing how the API naturally enables CPU-free gather/scatter halo exchange communication primitives in the Cabana/Kokkos performance portability framework, and through a performance comparison with Cray MPICH on the Frontier and Tuolumne supercomputers. Results from this evaluation show up to a 50% reduction in medium message latency in simple GPU ping-pong exchanges and a 28% speedup improvement when strong scaling a halo-exchange benchmark to 8,192 GPUs of the Frontier supercomputer. |
| title | Co-Design and Evaluation of a CPU-Free MPI GPU Communication Abstraction and Implementation |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2602.15356 |