BlendServe: Optimizing Offline Inference for Auto-regressive Large Models with Resource-aware Batching

Documenti analoghi

• ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving
  di: Qiao, Yifan, et al.
  Pubblicazione: (2024)
• Unleashing Scalable Context Parallelism for Foundation Models Pre-Training via FCP
  di: Zhao, Yilong, et al.
  Pubblicazione: (2026)
• Post-Training Sparse Attention with Double Sparsity
  di: Yang, Shuo, et al.
  Pubblicazione: (2024)
• Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
  di: Tang, Jiaming, et al.
  Pubblicazione: (2024)
• Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding
  di: Zhao, Yilong, et al.
  Pubblicazione: (2025)