DAK: Direct-Access-Enabled GPU Memory Offloading with Optimal Efficiency for LLM Inference

Documenti analoghi

• Breaking the Memory Wall: A Study of I/O Patterns and GPU Memory Utilization for Hybrid CPU-GPU Offloaded Optimizers
  di: Maurya, Avinash, et al.
  Pubblicazione: (2024)
• AQUA: Network-Accelerated Memory Offloading for LLMs in Scale-Up GPU Domains
  di: Kumar, Abhishek Vijaya, et al.
  Pubblicazione: (2024)
• NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference
  di: Jiang, Xuanlin, et al.
  Pubblicazione: (2024)
• Collaborative Inference for Large Models with Task Offloading and Early Exiting
  di: Xie, Zuan, et al.
  Pubblicazione: (2024)
• Enabling Dynamic Sparsity in Quantized LLM Inference
  di: Wang, Rongxiang, et al.
  Pubblicazione: (2025)