APEX: Asynchronous Parallel CPU-GPU Execution for Online LLM Inference on Constrained GPUs

Documenti analoghi

• Taming the Memory Footprint Crisis: System Design for Production Diffusion LLM Serving
  di: Fan, Jiakun, et al.
  Pubblicazione: (2025)
• SiPipe: Bridging the CPU-GPU Utilization Gap for Efficient Pipeline-Parallel LLM Inference
  di: He, Yongchao, et al.
  Pubblicazione: (2025)
• APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Serving
  di: Lin, Yi-Chien, et al.
  Pubblicazione: (2024)
• VDCores: Resource Decoupled Programming and Execution for Asynchronous GPU
  di: He, Zijian, et al.
  Pubblicazione: (2026)
• ParvaGPU: Efficient Spatial GPU Sharing for Large-Scale DNN Inference in Cloud Environments
  di: Lee, Munkyu, et al.
  Pubblicazione: (2024)