Boosting LLM Serving through Spatial-Temporal GPU Resource Sharing

Documenti analoghi

• gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
  di: Guo, Tianyu, et al.
  Pubblicazione: (2025)
• AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System
  di: Bai, Fengyao, et al.
  Pubblicazione: (2026)
• EcoServe: Enabling Cost-effective LLM Serving with Proactive Intra- and Inter-Instance Orchestration
  di: Du, Jiangsu, et al.
  Pubblicazione: (2025)
• MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
  di: Duan, Jiangfei, et al.
  Pubblicazione: (2024)
• TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inference
  di: Zhang, Hongbin, et al.
  Pubblicazione: (2025)