LiteCache: A Query Similarity-Driven, GPU-Centric KVCache Subsystem for Efficient LLM Inference

Documenti analoghi

• Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
  di: Zhao, Long, et al.
  Pubblicazione: (2026)
• BigMac: A Communication-Efficient Mixture-of-Experts Model Structure for Fast Training and Inference
  di: Jin, Zewen, et al.
  Pubblicazione: (2025)
• KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
  di: Wang, Jiahao, et al.
  Pubblicazione: (2025)
• AdaCluster: Adaptive Query-Key Clustering for Sparse Attention in Video Generation
  di: Tan, Haoyue, et al.
  Pubblicazione: (2026)
• Efficient Long-Context LLM Inference via KV Cache Clustering
  di: Hu, Jie, et al.
  Pubblicazione: (2025)