Cover Image

QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Imani, HamidReza, Peng, Jiaxin, Mohseni, Peiman, Amirany, Abdolah, El-Ghazawi, Tarek
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mixture of Experts with Mixture of Precisions for Tuning Quality of Service
by: Imani, HamidReza, et al.
Published: (2024)

DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance
by: Zhang, Yuning, et al.
Published: (2025)

Toward Cost-Efficient Serving of Mixture-of-Experts with Asynchrony
by: Wang, Shaoyu, et al.
Published: (2025)

QoE-oriented Dependent Task Scheduling under Multi-dimensional QoS Constraints over Distributed Networks
by: Fan, Xuwei, et al.
Published: (2023)

QONNECT: A QoS-Aware Orchestration System for Distributed Kubernetes Clusters
by: Aslan, Haci Ismail, et al.
Published: (2025)

QEdgeProxy: QoS-Aware Load Balancing for IoT Services in the Computing Continuum
by: Čilić, Ivan, et al.
Published: (2024)

Mercury: QoS-Aware Tiered Memory System
by: Lu, Jiaheng, et al.
Published: (2024)

QoS-aware Scheduling of Periodic Real-time Task Graphs on Heterogeneous Pre-occupied MECs
by: Shankar, Ashutosh, et al.
Published: (2025)

Leveraging knowledge-as-a-service (KaaS) for QoS-aware resource management in multi-user video transcoding
by: Costero, Luis, et al.
Published: (2024)

eBeeMetrics: An eBPF-based Library Framework for Feedback-free Observability of QoS Metrics
by: Ibnath, Muntaka, et al.
Published: (2026)

EAT: QoS-Aware Edge-Collaborative AIGC Task Scheduling via Attention-Guided Diffusion Reinforcement Learning
by: Xu, Zhifei, et al.
Published: (2025)

Large Language Model Aided QoS Prediction for Service Recommendation
by: Liu, Huiying, et al.
Published: (2024)

MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
by: Zhu, Ruidong, et al.
Published: (2025)

Fail-Closed Lowering of Resident KV Claims onto LLM Serving Runtimes
by: Stepanek, Lukas
Published: (2026)

PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
by: Hankendi, Can, et al.
Published: (2026)

MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing
by: Go, Seokjin, et al.
Published: (2025)

CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving
by: Zhao, Adrian, et al.
Published: (2026)

Autopoiesis: A Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamics
by: Jiang, Youhe, et al.
Published: (2026)

ExpertWeave: Efficiently Serving Expert-Specialized Fine-Tuned Adapters at Scale
by: Shi, Ge, et al.
Published: (2025)

MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
by: Duan, Jiangfei, et al.
Published: (2024)

QoS Aware Mixed-Criticality Task Scheduling in Vehicular Edge Cloud System
by: Sarkar, Suvarthi, et al.
Published: (2024)

Optimized Cloud Resource Allocation Using Genetic Algorithms for Energy Efficiency and QoS Assurance
by: Panggabean, Caroline, et al.
Published: (2025)

High Order Collaboration-Oriented Federated Graph Neural Network for Accurate QoS Prediction
by: Chen, Zehuan, et al.
Published: (2025)

FaaSMoE: A Serverless Framework for Multi-Tenant Mixture-of-Experts Serving
by: Wang, Minghe, et al.
Published: (2026)

RaMP: Runtime-Aware Megakernel Polymorphism for Mixture-of-Experts
by: Sharma, Vyom, et al.
Published: (2026)

FailLite: Failure-Resilient Model Serving for Resource-Constrained Edge Environments
by: Wu, Li, et al.
Published: (2025)

MoDM: Efficient Serving for Image Generation via Mixture-of-Diffusion Models
by: Xia, Yuchen, et al.
Published: (2025)

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
by: Hu, Jianmin, et al.
Published: (2025)

QoS-Aware Load Balancing in the Computing Continuum via Multi-Player Bandits
by: Čilić, Ivan, et al.
Published: (2025)

HAP: Hybrid Adaptive Parallelism for Efficient Mixture-of-Experts Inference
by: Lin, Haoran, et al.
Published: (2025)

DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs
by: Yao, Xiaozhe, et al.
Published: (2023)

Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving
by: Liu, Ziming, et al.
Published: (2025)

MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
by: Su, Zhaoyuan, et al.
Published: (2025)

Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts
by: Luo, Shuqing, et al.
Published: (2024)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
by: Wang, Zhibin, et al.
Published: (2025)

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
by: Mo, Zizhao, et al.
Published: (2025)

SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models
by: Du, Zhixu, et al.
Published: (2023)

Elastic Mixture of Rank-Wise Experts for Knowledge Reuse in Federated Fine-Tuning
by: Wu, Yebo, et al.
Published: (2025)

ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models
by: Singh, Gursimran, et al.
Published: (2025)

Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference
by: Sun, Xun, et al.
Published: (2026)