SageServe: Optimizing LLM Serving on Cloud Data Centers with Forecast Aware Auto-Scaling

Documenti analoghi

• Ensuring Fair LLM Serving Amid Diverse Applications
  di: Khan, Redwan Ibne Seraj, et al.
  Pubblicazione: (2024)
• Serving Heterogeneous LoRA Adapters in Distributed LLM Inference Systems
  di: Jaiswal, Shashwat, et al.
  Pubblicazione: (2025)
• Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Load Balancing
  di: Jain, Kunal, et al.
  Pubblicazione: (2024)
• Attention Enhanced Entity Recommendation for Intelligent Monitoring in Cloud Systems
  di: Hussain, Fiza, et al.
  Pubblicazione: (2025)
• Lean Attention: Hardware-Aware Scalable Attention Mechanism for the Decode-Phase of Transformers
  di: Sanovar, Rya, et al.
  Pubblicazione: (2024)