ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915814527991808 |
|---|---|
| author | Li, Haley Wang, Xinglu Feng, Cong Zuo, Chunxu Wang, Yanan Lo, Hei Cui, Yufei Wang, Bingji Cui, Duo Jing, Shuming Shan, Yizhou Xiong, Ying Wang, Jiannan Zhang, Yong Fan, Zhenan |
| author_facet | Li, Haley Wang, Xinglu Feng, Cong Zuo, Chunxu Wang, Yanan Lo, Hei Cui, Yufei Wang, Bingji Cui, Duo Jing, Shuming Shan, Yizhou Xiong, Ying Wang, Jiannan Zhang, Yong Fan, Zhenan |
| contents | As LLM deployments scale over more hardware, the probability of a single failure in a system increases significantly, and cloud operators must consider robust countermeasures to handle these inevitable failures. A common recovery approach is to simply restart the LLM serving instance; however, this is costly in model-as-a-service (MaaS) inference settings, where reloading model weights and recompiling computation graphs can introduce significant delays to incoming requests. We propose ReviveMoE, a method for rapid failure recovery in large-scale LLM deployments without restarting the serving instance. ReviveMoE is designed to support both the traditional LLM architecture, which collocates MoE and attention on the same hardware, and the disaggregated architectures, which separate MoE from attention. Integrated into Huawei Cloud's MaaS, ReviveMoE is built on top of Huawei's xDeepServe serving platform and the XCCL communications library. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_21140 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments Li, Haley Wang, Xinglu Feng, Cong Zuo, Chunxu Wang, Yanan Lo, Hei Cui, Yufei Wang, Bingji Cui, Duo Jing, Shuming Shan, Yizhou Xiong, Ying Wang, Jiannan Zhang, Yong Fan, Zhenan Distributed, Parallel, and Cluster Computing As LLM deployments scale over more hardware, the probability of a single failure in a system increases significantly, and cloud operators must consider robust countermeasures to handle these inevitable failures. A common recovery approach is to simply restart the LLM serving instance; however, this is costly in model-as-a-service (MaaS) inference settings, where reloading model weights and recompiling computation graphs can introduce significant delays to incoming requests. We propose ReviveMoE, a method for rapid failure recovery in large-scale LLM deployments without restarting the serving instance. ReviveMoE is designed to support both the traditional LLM architecture, which collocates MoE and attention on the same hardware, and the disaggregated architectures, which separate MoE from attention. Integrated into Huawei Cloud's MaaS, ReviveMoE is built on top of Huawei's xDeepServe serving platform and the XCCL communications library. |
| title | ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2602.21140 |