Tackling the Data-Parallel Load Balancing Bottleneck in LLM Serving: Practical Online Routing at Scale

Documenti analoghi

• A Universal Load Balancing Principle and Its Application to Large Language Model Serving
  di: Chen, Zixi, et al.
  Pubblicazione: (2026)
• S-HPLB: Efficient LLM Attention Serving via Sparsity-Aware Head Parallelism Load Balance
  di: Liu, Di, et al.
  Pubblicazione: (2026)
• Slice-Level Scheduling for High Throughput and Load Balanced LLM Serving
  di: Cheng, Ke, et al.
  Pubblicazione: (2024)
• gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
  di: Guo, Tianyu, et al.
  Pubblicazione: (2025)
• DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
  di: Yuan, Ying, et al.
  Pubblicazione: (2026)