Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMs

Documenti analoghi

• ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns
  di: Zhao, Ziyu, et al.
  Pubblicazione: (2026)
• Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
  di: Nrusimha, Aniruddha, et al.
  Pubblicazione: (2024)
• Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization
  di: Son, Seungwoo, et al.
  Pubblicazione: (2024)
• LQER: Low-Rank Quantization Error Reconstruction for LLMs
  di: Zhang, Cheng, et al.
  Pubblicazione: (2024)
• GLU Attention Improve Transformer
  di: Wang, Zehao
  Pubblicazione: (2025)