Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914556094185472 |
|---|---|
| author | Yu, Zhongkai Guan, Yue Yu, Zihao Zhou, Chenyang Hu, Zhengding Pei, Shuyi Kang, Yangwook Ding, Yufei Tsai, Po-An |
| author_facet | Yu, Zhongkai Guan, Yue Yu, Zihao Zhou, Chenyang Hu, Zhengding Pei, Shuyi Kang, Yangwook Ding, Yufei Tsai, Po-An |
| contents | Large-scale Mixture of Experts (MoE) Large Language Models (LLMs) have recently become the frontier open-weight models, achieving remarkable model capability similar to proprietary ones. But their random expert selection mechanism introduces significant data movement overhead that becomes the dominant bottleneck in multi-unit LLM serving systems.
To understand the patterns underlying this data movement, we conduct comprehensive data-movement-centric profiling across four state-of-the-art large-scale MoE models released in 2025 (200B-1000B) using over 24,000 requests spanning diverse workloads. We perform systematic analysis from both temporal and spatial perspectives and distill six key insights to guide the design of diverse serving systems. We verify these insights on both future wafer-scale GPU architectures and existing GPU systems. On wafer-scale GPUs, lightweight architectural modifications guided by our insights yield a 6.6$\times$ average speedup across four 200B--1000B models. On existing GPU systems, our insights drive the design of a prefill-aware expert placement algorithm that achieves up to 1.25$\times$ speedup on MoE computation. Our work presents the first comprehensive data-centric analysis of large-scale MoE models together with a concrete design study applying the learned lessons. Our profiling traces are publicly available at \href{https://huggingface.co/datasets/core12345/MoE_expert_selection_trace}{\textcolor{blue}{https://huggingface.co/datasets/core12345/MoE\_expert\_selection\_trace}}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_05497 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference Yu, Zhongkai Guan, Yue Yu, Zihao Zhou, Chenyang Hu, Zhengding Pei, Shuyi Kang, Yangwook Ding, Yufei Tsai, Po-An Distributed, Parallel, and Cluster Computing Artificial Intelligence Hardware Architecture Machine Learning Large-scale Mixture of Experts (MoE) Large Language Models (LLMs) have recently become the frontier open-weight models, achieving remarkable model capability similar to proprietary ones. But their random expert selection mechanism introduces significant data movement overhead that becomes the dominant bottleneck in multi-unit LLM serving systems. To understand the patterns underlying this data movement, we conduct comprehensive data-movement-centric profiling across four state-of-the-art large-scale MoE models released in 2025 (200B-1000B) using over 24,000 requests spanning diverse workloads. We perform systematic analysis from both temporal and spatial perspectives and distill six key insights to guide the design of diverse serving systems. We verify these insights on both future wafer-scale GPU architectures and existing GPU systems. On wafer-scale GPUs, lightweight architectural modifications guided by our insights yield a 6.6$\times$ average speedup across four 200B--1000B models. On existing GPU systems, our insights drive the design of a prefill-aware expert placement algorithm that achieves up to 1.25$\times$ speedup on MoE computation. Our work presents the first comprehensive data-centric analysis of large-scale MoE models together with a concrete design study applying the learned lessons. Our profiling traces are publicly available at \href{https://huggingface.co/datasets/core12345/MoE_expert_selection_trace}{\textcolor{blue}{https://huggingface.co/datasets/core12345/MoE\_expert\_selection\_trace}}. |
| title | Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence Hardware Architecture Machine Learning |
| url | https://arxiv.org/abs/2510.05497 |