DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving

Documenti analoghi

• Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
  di: Wang, Chao, et al.
  Pubblicazione: (2025)
• LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
  di: Wu, Bingyang, et al.
  Pubblicazione: (2024)
• Fast Distributed Inference Serving for Large Language Models
  di: Wu, Bingyang, et al.
  Pubblicazione: (2023)
• DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models
  di: Zhang, Zili, et al.
  Pubblicazione: (2024)
• BestServe: Serving Strategies with Optimal Goodput in Collocation and Disaggregation Architectures
  di: Hu, Xiannan, et al.
  Pubblicazione: (2025)