EcoServe: Enabling Cost-effective LLM Serving with Proactive Intra- and Inter-Instance Orchestration

Documenti analoghi

• AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System
  di: Bai, Fengyao, et al.
  Pubblicazione: (2026)
• TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inference
  di: Zhang, Hongbin, et al.
  Pubblicazione: (2025)
• gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
  di: Guo, Tianyu, et al.
  Pubblicazione: (2025)
• EcoServe: Designing Carbon-Aware AI Inference Systems
  di: Li, Yueying, et al.
  Pubblicazione: (2025)
• PipeMax: Enhancing Offline LLM Inference on Commodity GPU Servers
  di: Zhang, Hongbin, et al.
  Pubblicazione: (2026)