Image de couverture de livre

Accuracy Is Speed: Towards Long-Context-Aware Routing for Distributed LLM Serving

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yoshimura, Takeshi, van de Beek, Valentijn Dymphnus, Chiba, Tatsuhiro
Format:	Preprint
Publié:	2026
Sujets:	Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Speeding up Model Loading with fastsafetensors
par: Yoshimura, Takeshi, et autres
Publié: (2025)

SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
par: Zhou, Qihui, et autres
Publié: (2025)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
par: Hu, Cunchen, et autres
Publié: (2024)

Strata: Hierarchical Context Caching for Long Context Language Model Serving
par: Xie, Zhiqiang, et autres
Publié: (2025)

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference
par: Li, Suyi, et autres
Publié: (2024)

SageServe: Optimizing LLM Serving on Cloud Data Centers with Forecast Aware Auto-Scaling
par: Jaiswal, Shashwat, et autres
Publié: (2025)

GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
par: Du, Boxiao, et autres
Publié: (2026)

Aladdin: Joint Placement and Scaling for SLO-Aware LLM Serving
par: Nie, Chengyi, et autres
Publié: (2024)

Tackling the Data-Parallel Load Balancing Bottleneck in LLM Serving: Practical Online Routing at Scale
par: Bu, Tianci, et autres
Publié: (2026)

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
par: li, Fei, et autres
Publié: (2026)

Optimizing Long-context LLM Serving via Fine-grained Sequence Parallelism
par: Li, Cong, et autres
Publié: (2025)

APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Serving
par: Lin, Yi-Chien, et autres
Publié: (2024)

gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
par: Guo, Tianyu, et autres
Publié: (2025)

EdgeServing: Deadline-Aware Multi-DNN Serving at the Edge
par: Cao, Jiahe, et autres
Publié: (2026)

HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
par: Peng, You, et autres
Publié: (2026)

CascadeInfer: Length-Aware Scheduling of LLM Serving with Low Latency and Load Balancing
par: Yuan, Yitao, et autres
Publié: (2025)

DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
par: Ruan, Chaoyi, et autres
Publié: (2025)

MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
par: Duan, Jiangfei, et autres
Publié: (2024)

DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
par: Yuan, Ying, et autres
Publié: (2026)

Aragog: Just-in-Time Model Routing for Scalable Serving of Agentic Workflows
par: Dai, Yinwei, et autres
Publié: (2025)

S-HPLB: Efficient LLM Attention Serving via Sparsity-Aware Head Parallelism Load Balance
par: Liu, Di, et autres
Publié: (2026)

DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS
par: Basit, Omar, et autres
Publié: (2026)

EconoServe: Maximizing Multi-Resource Utilization with SLO Guarantees in LLM Serving
par: Shen, Haiying, et autres
Publié: (2024)

ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
par: Jiang, Youhe, et autres
Publié: (2025)

Predictable LLM Serving on GPU Clusters
par: Darzi, Erfan, et autres
Publié: (2025)

Loki: A System for Serving ML Inference Pipelines with Hardware and Accuracy Scaling
par: Ahmad, Sohaib, et autres
Publié: (2024)

HydraServe: Minimizing Cold Start Latency for Serverless LLM Serving in Public Clouds
par: Lou, Chiheng, et autres
Publié: (2025)

TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
par: Liu, Dong, et autres
Publié: (2025)

DiffServe: Efficiently Serving Text-to-Image Diffusion Models with Query-Aware Model Scaling
par: Ahmad, Sohaib, et autres
Publié: (2024)

SIMPLE: Disaggregating Sampling from GPU Inference into a Decision Plane for Faster Distributed LLM Serving
par: Zhao, Bohan, et autres
Publié: (2025)

Cloud Native System for LLM Inference Serving
par: Xu, Minxian, et autres
Publié: (2025)

Fast State Restoration in LLM Serving with HCache
par: Gao, Shiwei, et autres
Publié: (2024)

EcoServe: Enabling Cost-effective LLM Serving with Proactive Intra- and Inter-Instance Orchestration
par: Du, Jiangsu, et autres
Publié: (2025)

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
par: Hu, Jianmin, et autres
Publié: (2025)

eLLM: Elastic Memory Management Framework for Efficient LLM Serving
par: Xu, Jiale, et autres
Publié: (2025)

LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
par: Wu, Bingyang, et autres
Publié: (2024)

Jenga: Effective Memory Management for Serving LLM with Heterogeneity
par: Zhang, Chen, et autres
Publié: (2025)

RouterWise: Joint Resource Allocation and Routing for Latency-Aware Multi-Model LLM Serving
par: Kasnavieh, Hossein Hosseini, et autres
Publié: (2026)

Argus: Token Aware Distributed LLM Inference Optimization
par: Wu, Panlong, et autres
Publié: (2025)

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
par: He, Yiyuan, et autres
Publié: (2025)