Mixture of Attention Spans: Optimizing LLM Inference Efficiency with Heterogeneous Sliding-Window Lengths

Documenti analoghi

• Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs
  di: Liu, Enshu, et al.
  Pubblicazione: (2024)
• Evaluating Quantized Large Language Models
  di: Li, Shiyao, et al.
  Pubblicazione: (2024)
• FrameFusion: Combining Similarity and Importance for Video Token Reduction on Large Vision Language Models
  di: Fu, Tianyu, et al.
  Pubblicazione: (2024)
• ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation
  di: Zhao, Tianchen, et al.
  Pubblicazione: (2024)
• CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarios
  di: Wang, Luning, et al.
  Pubblicazione: (2024)