Fast Heterogeneous Serving: Scalable Mixed-Scale LLM Allocation for SLO-Constrained Inference

Documenti analoghi

• Green-LLM: Optimal Workload Allocation for Environmentally-Aware Distributed Inference
  di: Cheng, Jiaming, et al.
  Pubblicazione: (2025)
• Two-Stage Distributionally Robust Edge Node Placement Under Endogenous Demand Uncertainty
  di: Cheng, Jiaming, et al.
  Pubblicazione: (2024)
• CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving
  di: Liu, Yuhan, et al.
  Pubblicazione: (2023)
• Hybrid-Task Meta-Learning: A GNN Approach for Scalable and Transferable Bandwidth Allocation
  di: Hao, Xin, et al.
  Pubblicazione: (2023)
• HetCCL: Enabling Collective Communication For Mixed-Vendor Heterogeneous Clusters
  di: Wang, Yuejie, et al.
  Pubblicazione: (2026)