Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909975818797056 |
|---|---|
| author | Pan, Xinglin Shi, Shaohuai Lin, Wenxiang Wang, Yuxin Tang, Zhenheng Wang, Wei Chu, Xiaowen |
| author_facet | Pan, Xinglin Shi, Shaohuai Lin, Wenxiang Wang, Yuxin Tang, Zhenheng Wang, Wei Chu, Xiaowen |
| contents | The mixture-of-experts (MoE) architecture scales model size with sublinear computational increase but suffers from memory-intensive inference due to KV caches and sparse expert activation. Recent disaggregated expert parallelism (DEP) distributes attention and experts to dedicated GPU groups but lacks support for shared experts and efficient task scheduling, limiting performance.
We propose FinDEP, a fine-grained task scheduling algorithm for DEP that maximizes task overlap to improve MoE inference throughput. FinDEP introduces three innovations: 1) partitioning computation/communication into smaller tasks for fine-grained pipelining, 2) formulating a scheduling optimization supporting variable granularity and ordering, and 3) developing an efficient solver for this large search space.
Experiments on four GPU systems with DeepSeek-V2 and Qwen3-MoE show FinDEP improves throughput by up to 1.61x over prior methods, achieving up to 1.24x speedup on a 32-GPU system. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_21487 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism Pan, Xinglin Shi, Shaohuai Lin, Wenxiang Wang, Yuxin Tang, Zhenheng Wang, Wei Chu, Xiaowen Distributed, Parallel, and Cluster Computing Artificial Intelligence The mixture-of-experts (MoE) architecture scales model size with sublinear computational increase but suffers from memory-intensive inference due to KV caches and sparse expert activation. Recent disaggregated expert parallelism (DEP) distributes attention and experts to dedicated GPU groups but lacks support for shared experts and efficient task scheduling, limiting performance. We propose FinDEP, a fine-grained task scheduling algorithm for DEP that maximizes task overlap to improve MoE inference throughput. FinDEP introduces three innovations: 1) partitioning computation/communication into smaller tasks for fine-grained pipelining, 2) formulating a scheduling optimization supporting variable granularity and ordering, and 3) developing an efficient solver for this large search space. Experiments on four GPU systems with DeepSeek-V2 and Qwen3-MoE show FinDEP improves throughput by up to 1.61x over prior methods, achieving up to 1.24x speedup on a 32-GPU system. |
| title | Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence |
| url | https://arxiv.org/abs/2512.21487 |