SpecOffload: Unlocking Latent GPU Capacity for LLM Inference on Resource-Constrained Devices

Documenti analoghi

• HCInfer: An Efficient Inference System via Error Compensation for Resource-Constrained Devices
  di: Xu, Shen, et al.
  Pubblicazione: (2026)
• Camel: Energy-Aware LLM Inference on Resource-Constrained Devices
  di: Xu, Hao, et al.
  Pubblicazione: (2025)
• NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference
  di: Jiang, Xuanlin, et al.
  Pubblicazione: (2024)
• SpecInF: Exploiting Idle GPU Resources in Distributed DL Training via Speculative Inference Filling
  di: Lv, Cunchi, et al.
  Pubblicazione: (2025)
• DAK: Direct-Access-Enabled GPU Memory Offloading with Optimal Efficiency for LLM Inference
  di: Lin, Shouxu, et al.
  Pubblicazione: (2026)