Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs

Documenti analoghi

• DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
  di: Zhong, Yinmin, et al.
  Pubblicazione: (2024)
• Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
  di: Wang, Chao, et al.
  Pubblicazione: (2025)
• Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving
  di: Shi, Xiaoxiang, et al.
  Pubblicazione: (2025)
• PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
  di: Woo, Sunghyeon, et al.
  Pubblicazione: (2026)
• InfiniLoRA: Disaggregated Multi-LoRA Serving for Large Language Models
  di: Chen, Hongyu, et al.
  Pubblicazione: (2026)