CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference

Documenti analoghi

• GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
  di: Du, Boxiao, et al.
  Pubblicazione: (2026)
• InfiniLoRA: Disaggregated Multi-LoRA Serving for Large Language Models
  di: Chen, Hongyu, et al.
  Pubblicazione: (2026)
• S-LoRA: Serving Thousands of Concurrent LoRA Adapters
  di: Sheng, Ying, et al.
  Pubblicazione: (2023)
• Serving Heterogeneous LoRA Adapters in Distributed LLM Inference Systems
  di: Jaiswal, Shashwat, et al.
  Pubblicazione: (2025)
• MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
  di: Hu, Cunchen, et al.
  Pubblicazione: (2024)