FUSCO: High-Performance Distributed Data Shuffling via Transformation-Communication Fusion
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909976333647872 |
|---|---|
| author | Zhu, Zhuoran Zhu, Chunyang Lin, Hao Fu, Xu Zhou, Yiming Zhang, Quanlu Li, Zhenhua Qian, Feng Yu, Chao Li, Boxun Dai, Guohao Wang, Yu |
| author_facet | Zhu, Zhuoran Zhu, Chunyang Lin, Hao Fu, Xu Zhou, Yiming Zhang, Quanlu Li, Zhenhua Qian, Feng Yu, Chao Li, Boxun Dai, Guohao Wang, Yu |
| contents | Large-scale Mixture-of-Experts (MoE) models rely on \emph{expert parallelism} for efficient training and inference, which splits experts across devices and necessitates distributed data shuffling to route each token to its assigned experts. However, existing communication libraries handle this shuffling poorly; its overhead can account for over half of end-to-end runtime. We present FUSCO, an MoE-friendly communication library that achieves efficient and lightweight data shuffling through fused data transformation and communication, based on the key observation that MoE's expert-major data layout conflicts with the device-major layout expected by communication operations. FUSCO captures the fine-grained data layout, which is then interpreted by a pipelined communication engine that performs the required shuffling efficiently along the communication path. Lightweight planning and load-balancing mechanisms complement the engine by eliminating redundant communication and dispersing traffic. Evaluations on representative benchmarks illustrate that FUSCO achieves up to 3.84$\times$ and 2.01$\times$ speedups over NCCL and DeepEP (the state-of-the-art MoE communication library), respectively. In end-to-end MoE tasks, compared to NCCL and DeepEP, FUSCO reduces the training latency by 1.17-1.39$\times$ and 1.10-1.19$\times$, and lowers the first-token generation latency in inference by 1.09-1.25$\times$ and 1.06-1.16$\times$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_22036 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FUSCO: High-Performance Distributed Data Shuffling via Transformation-Communication Fusion Zhu, Zhuoran Zhu, Chunyang Lin, Hao Fu, Xu Zhou, Yiming Zhang, Quanlu Li, Zhenhua Qian, Feng Yu, Chao Li, Boxun Dai, Guohao Wang, Yu Distributed, Parallel, and Cluster Computing Large-scale Mixture-of-Experts (MoE) models rely on \emph{expert parallelism} for efficient training and inference, which splits experts across devices and necessitates distributed data shuffling to route each token to its assigned experts. However, existing communication libraries handle this shuffling poorly; its overhead can account for over half of end-to-end runtime. We present FUSCO, an MoE-friendly communication library that achieves efficient and lightweight data shuffling through fused data transformation and communication, based on the key observation that MoE's expert-major data layout conflicts with the device-major layout expected by communication operations. FUSCO captures the fine-grained data layout, which is then interpreted by a pipelined communication engine that performs the required shuffling efficiently along the communication path. Lightweight planning and load-balancing mechanisms complement the engine by eliminating redundant communication and dispersing traffic. Evaluations on representative benchmarks illustrate that FUSCO achieves up to 3.84$\times$ and 2.01$\times$ speedups over NCCL and DeepEP (the state-of-the-art MoE communication library), respectively. In end-to-end MoE tasks, compared to NCCL and DeepEP, FUSCO reduces the training latency by 1.17-1.39$\times$ and 1.10-1.19$\times$, and lowers the first-token generation latency in inference by 1.09-1.25$\times$ and 1.06-1.16$\times$. |
| title | FUSCO: High-Performance Distributed Data Shuffling via Transformation-Communication Fusion |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2512.22036 |