Copertina

Toward Cost-Efficient Serving of Mixture-of-Experts with Asynchrony

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Shaoyu, He, Guangrong, Kim, Geon-Woo, Zhou, Yanqi, Park, Seo Jin
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving
di: Zhao, Adrian, et al.
Pubblicazione: (2026)

Consensus In Asynchrony
di: Klianev, Ivan
Pubblicazione: (2026)

Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving
di: Liu, Ziming, et al.
Pubblicazione: (2025)

BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models
di: Hu, Bodun, et al.
Pubblicazione: (2024)

ExpertWeave: Efficiently Serving Expert-Specialized Fine-Tuned Adapters at Scale
di: Shi, Ge, et al.
Pubblicazione: (2025)

MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
di: Zhu, Ruidong, et al.
Pubblicazione: (2025)

Tolerance to Asynchrony in Algorithms for Multiplication and Modulo
di: Gupta, Arya Tanmay, et al.
Pubblicazione: (2023)

Asynchrony-Resilient Sleepy Total-Order Broadcast Protocols
di: D'Amato, Francesco, et al.
Pubblicazione: (2023)

Harpagon: Minimizing DNN Serving Cost via Efficient Dispatching, Scheduling and Splitting
di: Zhao, Zhixin, et al.
Pubblicazione: (2024)

QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration
di: Imani, HamidReza, et al.
Pubblicazione: (2025)

MoDM: Efficient Serving for Image Generation via Mixture-of-Diffusion Models
di: Xia, Yuchen, et al.
Pubblicazione: (2025)

OMEGA: A Low-Latency GNN Serving System for Large Graphs
di: Kim, Geon-Woo, et al.
Pubblicazione: (2025)

ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts
di: Luo, Shuqing, et al.
Pubblicazione: (2024)

On the Decidability of Distributed Tasks with Output Sets under Asynchrony and Any Number of Crashes
di: Albouy, Timothé, et al.
Pubblicazione: (2026)

PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
di: Hankendi, Can, et al.
Pubblicazione: (2026)

HAP: Hybrid Adaptive Parallelism for Efficient Mixture-of-Experts Inference
di: Lin, Haoran, et al.
Pubblicazione: (2025)

MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing
di: Go, Seokjin, et al.
Pubblicazione: (2025)

Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

Tolerance to Asynchrony of an Algorithm for Gathering Myopic Robots on an Infinite Triangular Grid
di: Gupta, Arya Tanmay, et al.
Pubblicazione: (2023)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
di: Wang, Zhibin, et al.
Pubblicazione: (2025)

Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
di: Kim, Kihyun, et al.
Pubblicazione: (2025)

FaaSMoE: A Serverless Framework for Multi-Tenant Mixture-of-Experts Serving
di: Wang, Minghe, et al.
Pubblicazione: (2026)

ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models
di: Singh, Gursimran, et al.
Pubblicazione: (2025)

EcoServe: Enabling Cost-effective LLM Serving with Proactive Intra- and Inter-Instance Orchestration
di: Du, Jiangsu, et al.
Pubblicazione: (2025)

BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization
di: Jiang, Youhe, et al.
Pubblicazione: (2026)

SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models
di: Du, Zhixu, et al.
Pubblicazione: (2023)

DiffServe: Efficiently Serving Text-to-Image Diffusion Models with Query-Aware Model Scaling
di: Ahmad, Sohaib, et al.
Pubblicazione: (2024)

UELLM: A Unified and Efficient Approach for LLM Inference Serving
di: He, Yiyuan, et al.
Pubblicazione: (2024)

MoLink: Distributed and Efficient Serving Framework for Large Models
di: Jin, Lewei, et al.
Pubblicazione: (2025)

Efficient Multi-round LLM Inference over Disaggregated Serving
di: He, Wenhao, et al.
Pubblicazione: (2026)

MixServe: An Automatic Distributed Serving System for MoE Models with Hybrid Parallelism Based on Fused Communication Algorithm
di: Zhou, Bowen, et al.
Pubblicazione: (2026)

HydraServe: Minimizing Cold Start Latency for Serverless LLM Serving in Public Clouds
di: Lou, Chiheng, et al.
Pubblicazione: (2025)

GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
di: Du, Boxiao, et al.
Pubblicazione: (2026)

Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity
di: Griggs, Tyler, et al.
Pubblicazione: (2024)

Communication-Efficient Serving for Video Diffusion Models with Latent Parallelism
di: Wu, Zhiyuan, et al.
Pubblicazione: (2025)

eLLM: Elastic Memory Management Framework for Efficient LLM Serving
di: Xu, Jiale, et al.
Pubblicazione: (2025)

Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation
di: Kim, Joon Ha, et al.
Pubblicazione: (2026)

HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
di: Peng, You, et al.
Pubblicazione: (2026)

FREESH: Fair, Resource- and Energy-Efficient Scheduling for LLM Serving on Heterogeneous GPUs
di: He, Xuan, et al.
Pubblicazione: (2025)