SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layers

Documenti analoghi

• KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embedding
  di: Shi, Luohe, et al.
  Pubblicazione: (2025)
• XQuant: Achieving Ultra-Low Bit KV Cache Quantization with Cross-Layer Compression
  di: Yang, Haoqi, et al.
  Pubblicazione: (2025)
• Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios
  di: Shi, Luohe, et al.
  Pubblicazione: (2025)
• Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption
  di: Shi, Luohe, et al.
  Pubblicazione: (2024)
• Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models
  di: Shi, Luohe, et al.
  Pubblicazione: (2024)