Imagen de Portada

MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Duan, Jiangfei, Lu, Runyu, Duanmu, Haojie, Li, Xiuhong, Zhang, Xingcheng, Lin, Dahua, Stoica, Ion, Zhang, Hao
Formato:	Preprint
Publicado:	2024
Materias:	Distributed, Parallel, and Cluster Computing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decoding
por: Chen, Jiefei, et al.
Publicado: (2026)

MuxTune: Efficient Multi-Task LLM Fine-Tuning in Multi-Tenant Datacenters via Spatial-Temporal Backbone Multiplexing
por: Xue, Chunyu, et al.
Publicado: (2026)

Boosting LLM Serving through Spatial-Temporal GPU Resource Sharing
por: Lin, Zejia, et al.
Publicado: (2025)

Jenga: Effective Memory Management for Serving LLM with Heterogeneity
por: Zhang, Chen, et al.
Publicado: (2025)

OServe: Accelerating LLM Serving via Spatial-Temporal Workload Orchestration
por: Jiang, Youhe, et al.
Publicado: (2026)

Past-Future Scheduler for LLM Serving under SLA Guarantees
por: Gong, Ruihao, et al.
Publicado: (2025)

ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving
por: Qiao, Yifan, et al.
Publicado: (2024)

Locality-aware Fair Scheduling in LLM Serving
por: Cao, Shiyi, et al.
Publicado: (2025)

RollMux: Phase-Level Multiplexing for Disaggregated RL Post-Training
por: Wu, Tianyuan, et al.
Publicado: (2025)

SkyServe: Serving AI Models across Regions and Clouds with Spot Instances
por: Mao, Ziming, et al.
Publicado: (2024)

EcoServe: Enabling Cost-effective LLM Serving with Proactive Intra- and Inter-Instance Orchestration
por: Du, Jiangsu, et al.
Publicado: (2025)

Foundry: Template-Based CUDA Graph Context Materialization for Fast LLM Serving Cold Start
por: Liu, Xueshen, et al.
Publicado: (2026)

TetriServe: Efficient DiT Serving for Heterogeneous Image Generation
por: Lu, Runyu, et al.
Publicado: (2025)

AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System
por: Bai, Fengyao, et al.
Publicado: (2026)

DeepServe: Serverless Large Language Model Serving at Scale
por: Hu, Junhao, et al.
Publicado: (2025)

DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
por: Ruan, Chaoyi, et al.
Publicado: (2025)

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference
por: Li, Suyi, et al.
Publicado: (2024)

TridentServe: A Stage-level Serving System for Diffusion Pipelines
por: Xia, Yifei, et al.
Publicado: (2025)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
por: Hu, Cunchen, et al.
Publicado: (2024)

EconoServe: Maximizing Multi-Resource Utilization with SLO Guarantees in LLM Serving
por: Shen, Haiying, et al.
Publicado: (2024)

ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
por: Jiang, Youhe, et al.
Publicado: (2025)

EdgeServing: Deadline-Aware Multi-DNN Serving at the Edge
por: Cao, Jiahe, et al.
Publicado: (2026)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
por: Zhong, Yinmin, et al.
Publicado: (2024)

HydraServe: Minimizing Cold Start Latency for Serverless LLM Serving in Public Clouds
por: Lou, Chiheng, et al.
Publicado: (2025)

SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
por: Zhou, Qihui, et al.
Publicado: (2025)

Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity
por: Griggs, Tyler, et al.
Publicado: (2024)

eLLM: Elastic Memory Management Framework for Efficient LLM Serving
por: Xu, Jiale, et al.
Publicado: (2025)

Predictable LLM Serving on GPU Clusters
por: Darzi, Erfan, et al.
Publicado: (2025)

Autellix: An Efficient Serving Engine for LLM Agents as General Programs
por: Luo, Michael, et al.
Publicado: (2025)

gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
por: Guo, Tianyu, et al.
Publicado: (2025)

GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
por: Du, Boxiao, et al.
Publicado: (2026)

SageServe: Optimizing LLM Serving on Cloud Data Centers with Forecast Aware Auto-Scaling
por: Jaiswal, Shashwat, et al.
Publicado: (2025)

vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving
por: Xu, Jiale, et al.
Publicado: (2024)

Nitsum: Serving Tiered LLM Requests with Adaptive Tensor Parallelism
por: Srivatsa, Vikranth, et al.
Publicado: (2026)

PROSERVE: Unified Multi-Priority Request Scheduling for LLM Serving
por: Huang, Weizhe, et al.
Publicado: (2025)

EPD-Serve: A Flexible Multimodal EPD Disaggregation Inference Serving System On Ascend
por: Bai, Fan, et al.
Publicado: (2026)

TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity
por: Lai, Ruiqi, et al.
Publicado: (2025)

Cloud Native System for LLM Inference Serving
por: Xu, Minxian, et al.
Publicado: (2025)

Fast State Restoration in LLM Serving with HCache
por: Gao, Shiwei, et al.
Publicado: (2024)

Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances
por: Duan, Jiangfei, et al.
Publicado: (2024)