Scalable Processing-Near-Memory for 1M-Token LLM Inference: CXL-Enabled KV-Cache Management Beyond GPU Limits

Documenti analoghi

• Cosmos: A CXL-Based Full In-Memory System for Approximate Nearest Neighbor Search
  di: Ko, Seoyoung, et al.
  Pubblicazione: (2025)
• TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
  di: Yoon, Dongha, et al.
  Pubblicazione: (2025)
• KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction
  di: Kim, Jang-Hyun, et al.
  Pubblicazione: (2025)
• InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding
  di: Kim, Minsoo, et al.
  Pubblicazione: (2025)
• No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization
  di: Yang, June Yong, et al.
  Pubblicazione: (2024)