KVPR: Efficient LLM Inference with I/O-Aware KV Cache Partial Recomputation

Documenti analoghi

• Striking the Right Balance between Compute and Copy: Improving LLM Inferencing Under Speculative Decoding
  di: Ramachandran, Arun, et al.
  Pubblicazione: (2025)
• MobiZO: Enabling Efficient LLM Fine-Tuning at the Edge via Inference Engines
  di: Gao, Lei, et al.
  Pubblicazione: (2024)
• DUAL-BLADE: Dual-Path NVMe-Direct KV-Cache Offloading for Edge LLM Inference
  di: Jeong, Bodon, et al.
  Pubblicazione: (2026)
• OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
  di: Zhou, Zhongzhu, et al.
  Pubblicazione: (2026)
• KVDirect: Distributed Disaggregated LLM Inference
  di: Chen, Shiyang, et al.
  Pubblicazione: (2024)