XShare: Collaborative in-Batch Expert Sharing for Faster MoE Inference
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910014600380416 |
|---|---|
| author | Vankov, Daniil Ivkin, Nikita Ulrich, Kyle Song, Xiang Khetan, Ashish Karypis, George |
| author_facet | Vankov, Daniil Ivkin, Nikita Ulrich, Kyle Song, Xiang Khetan, Ashish Karypis, George |
| contents | Mixture-of-Experts (MoE) architectures are increasingly used to efficiently scale large language models. However, in production inference, request batching and speculative decoding significantly amplify expert activation, eroding these efficiency benefits. We address this issue by modeling batch-aware expert selection as a modular optimization problem and designing efficient greedy algorithms for different deployment settings. The proposed method, namely XShare, requires no retraining and dynamically adapts to each batch by maximizing the total gating score of selected experts. It reduces expert activation by up to 30% under standard batching, cuts peak GPU load by up to 3x in expert-parallel deployments, and achieves up to 14% throughput gains in speculative decoding via hierarchical, correlation-aware expert selection even if requests in a batch drawn from heterogeneous datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_07265 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | XShare: Collaborative in-Batch Expert Sharing for Faster MoE Inference Vankov, Daniil Ivkin, Nikita Ulrich, Kyle Song, Xiang Khetan, Ashish Karypis, George Machine Learning Artificial Intelligence I.2.7 Mixture-of-Experts (MoE) architectures are increasingly used to efficiently scale large language models. However, in production inference, request batching and speculative decoding significantly amplify expert activation, eroding these efficiency benefits. We address this issue by modeling batch-aware expert selection as a modular optimization problem and designing efficient greedy algorithms for different deployment settings. The proposed method, namely XShare, requires no retraining and dynamically adapts to each batch by maximizing the total gating score of selected experts. It reduces expert activation by up to 30% under standard batching, cuts peak GPU load by up to 3x in expert-parallel deployments, and achieves up to 14% throughput gains in speculative decoding via hierarchical, correlation-aware expert selection even if requests in a batch drawn from heterogeneous datasets. |
| title | XShare: Collaborative in-Batch Expert Sharing for Faster MoE Inference |
| topic | Machine Learning Artificial Intelligence I.2.7 |
| url | https://arxiv.org/abs/2602.07265 |