Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System

Documenti analoghi

• Reimagining Memory Access for LLM Inference: Compression-Aware Memory Controller Design
  di: Xie, Rui, et al.
  Pubblicazione: (2025)
• Making Strong Error-Correcting Codes Work Effectively for HBM in AI Inference
  di: Xie, Rui, et al.
  Pubblicazione: (2025)
• Breaking the HBM Bit Cost Barrier: Domain-Specific ECC for AI Inference Infrastructure
  di: Xie, Rui, et al.
  Pubblicazione: (2025)
• TRACE: Unlocking Effective CXL Bandwidth via Lossless Compression and Precision Scaling
  di: Xie, Rui, et al.
  Pubblicazione: (2025)
• Adaptive Cache Pollution Control for Large Language Model Inference Workloads Using Temporal CNN-Based Prediction and Priority-Aware Replacement
  di: Liu, Songze, et al.
  Pubblicazione: (2025)