XQuant: Achieving Ultra-Low Bit KV Cache Quantization with Cross-Layer Compression

Documenti analoghi

• KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embedding
  di: Shi, Luohe, et al.
  Pubblicazione: (2025)
• SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layers
  di: Tang, Zicong, et al.
  Pubblicazione: (2025)
• DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compression
  di: Zhao, Yi, et al.
  Pubblicazione: (2025)
• Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption
  di: Shi, Luohe, et al.
  Pubblicazione: (2024)
• Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios
  di: Shi, Luohe, et al.
  Pubblicazione: (2025)