Image de couverture de livre

Equinox: Holistic Fair Scheduling in Serving Large Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wei, Zhixiang, Yen, James, Chen, Jingyi, Zhang, Ziyang, Huang, Zhibai, Chen, Chen, Yu, Xingzi, Gu, Yicheng, Wu, Chenggang, Wang, Yun, Xia, Mingyuan, Wu, Jie, Wang, Hao, Qi, Zhengwei
Format:	Preprint
Publié:	2025
Sujets:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

FREESH: Fair, Resource- and Energy-Efficient Scheduling for LLM Serving on Heterogeneous GPUs
par: He, Xuan, et autres
Publié: (2025)

Locality-aware Fair Scheduling in LLM Serving
par: Cao, Shiyi, et autres
Publié: (2025)

A Predictive and Synergistic Two-Layer Scheduling Framework for LLM Serving
par: Zhang, Yue, et autres
Publié: (2025)

WWW.Serve: Interconnecting Global LLM Services through Decentralization
par: Wang, Huanyu, et autres
Publié: (2026)

Past-Future Scheduler for LLM Serving under SLA Guarantees
par: Gong, Ruihao, et autres
Publié: (2025)

HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
par: Peng, You, et autres
Publié: (2026)

HydraInfer: Hybrid Disaggregated Scheduling for Multimodal Large Language Model Serving
par: Dong, Xianzhe, et autres
Publié: (2025)

KCES: A Workflow Containerization Scheduling Scheme Under Cloud-Edge Collaboration Framework
par: Shan, Chenggang, et autres
Publié: (2024)

DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
par: Ruan, Chaoyi, et autres
Publié: (2025)

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference
par: Li, Suyi, et autres
Publié: (2024)

Communication-Efficient Serving for Video Diffusion Models with Latent Parallelism
par: Wu, Zhiyuan, et autres
Publié: (2025)

CascadeInfer: Length-Aware Scheduling of LLM Serving with Low Latency and Load Balancing
par: Yuan, Yitao, et autres
Publié: (2025)

Slice-Level Scheduling for High Throughput and Load Balanced LLM Serving
par: Cheng, Ke, et autres
Publié: (2024)

EcoServe: Enabling Cost-effective LLM Serving with Proactive Intra- and Inter-Instance Orchestration
par: Du, Jiangsu, et autres
Publié: (2025)

OTAS: An Elastic Transformer Serving System via Token Adaptation
par: Chen, Jinyu, et autres
Publié: (2024)

It Takes Two to Tango: Serverless Workflow Serving via Bilaterally Engaged Resource Adaptation
par: Wu, Jing, et autres
Publié: (2025)

CALVO: Improve Serving Efficiency for LLM Inferences with Intense Network Demands
par: Wang, Weiye, et autres
Publié: (2026)

Harpagon: Minimizing DNN Serving Cost via Efficient Dispatching, Scheduling and Splitting
par: Zhao, Zhixin, et autres
Publié: (2024)

RingAda: Pipelining Large Model Fine-Tuning on Edge Devices with Scheduled Layer Unfreezing
par: Li, Liang, et autres
Publié: (2025)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
par: Hu, Cunchen, et autres
Publié: (2024)

TridentServe: A Stage-level Serving System for Diffusion Pipelines
par: Xia, Yifei, et autres
Publié: (2025)

Cortex: Workflow-Aware Resource Pooling and Scheduling for Agentic Serving
par: Pagonas, Nikos, et autres
Publié: (2025)

PROSERVE: Unified Multi-Priority Request Scheduling for LLM Serving
par: Huang, Weizhe, et autres
Publié: (2025)

DeepServe: Serverless Large Language Model Serving at Scale
par: Hu, Junhao, et autres
Publié: (2025)

OmniInfer: System-Wide Acceleration Techniques for Optimizing LLM Serving Throughput and Latency
par: Wang, Jun, et autres
Publié: (2025)

Multi-stage Flow Scheduling for LLM Serving
par: Sun, Yijun, et autres
Publié: (2026)

GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
par: Du, Boxiao, et autres
Publié: (2026)

AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System
par: Bai, Fengyao, et autres
Publié: (2026)

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
par: He, Yiyuan, et autres
Publié: (2025)

Adaptive Resource Allocation for Workflow Containerization on Kubernetes
par: Shan, Chenggang, et autres
Publié: (2023)

Equinox: Decentralized Scheduling for Hardware-Aware Orbital Intelligence
par: Erol, Ansel Kaplan, et autres
Publié: (2026)

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
par: Chen, Xing, et autres
Publié: (2025)

Fast State Restoration in LLM Serving with HCache
par: Gao, Shiwei, et autres
Publié: (2024)

SneakPeek: Data-Aware Model Selection and Scheduling for Inference Serving on the Edge
par: Wolfrath, Joel, et autres
Publié: (2025)

Megha: Decentralized Global Fair Scheduling for Federated Clusters
par: Thiyyakat, Meghana, et autres
Publié: (2021)

TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
par: Bian, Zhuohang, et autres
Publié: (2026)

OOCO: Latency-disaggregated Architecture for Online-Offline Co-locate LLM Serving
par: Wu, Siyu, et autres
Publié: (2025)

Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Serving
par: Ding, Jianru, et autres
Publié: (2026)

Boosting LLM Serving through Spatial-Temporal GPU Resource Sharing
par: Lin, Zejia, et autres
Publié: (2025)

Symphony: Optimized DNN Model Serving using Deferred Batch Scheduling
par: Chen, Lequn, et autres
Publié: (2023)