BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving

Documenti analoghi

• UELLM: A Unified and Efficient Approach for LLM Inference Serving
  di: He, Yiyuan, et al.
  Pubblicazione: (2024)
• DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
  di: Liao, Junhan, et al.
  Pubblicazione: (2025)
• BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
  di: Hu, Jianmin, et al.
  Pubblicazione: (2025)
• Cloud Native System for LLM Inference Serving
  di: Xu, Minxian, et al.
  Pubblicazione: (2025)
• Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
  di: Wu, Jingfeng, et al.
  Pubblicazione: (2025)