QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache

Documenti analoghi

• XQuant: Breaking the Memory Wall for LLM Inference with KV Cache Rematerialization
  di: Tomar, Aditya, et al.
  Pubblicazione: (2025)
• KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
  di: Hooper, Coleman, et al.
  Pubblicazione: (2024)
• SPEED: Speculative Pipelined Execution for Efficient Decoding
  di: Hooper, Coleman, et al.
  Pubblicazione: (2023)
• AI and Memory Wall
  di: Gholami, Amir, et al.
  Pubblicazione: (2024)
• Arbitrage: Efficient Reasoning via Advantage-Aware Speculation
  di: Maheswaran, Monishwaran, et al.
  Pubblicazione: (2025)