Leveraging Speculative Sampling and KV-Cache Optimizations Together for Generative AI using OpenVINO

Documenti analoghi

• ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression
  di: Liu, Guangda, et al.
  Pubblicazione: (2024)
• ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching
  di: Zhao, Youpeng, et al.
  Pubblicazione: (2024)
• QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead
  di: Zandieh, Amir, et al.
  Pubblicazione: (2024)
• Dual-Signal Adaptive KV-Cache Optimization for Long-Form Video Understanding in Vision-Language Models
  di: Sai, Vishnu, et al.
  Pubblicazione: (2026)
• Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management
  di: Zhang, Hang, et al.
  Pubblicazione: (2025)