Depth Registers Unlock W4A4 on SwiGLU: A Reader/Generator Decomposition

Documenti analoghi

• Dual Path Attribution: Efficient Attribution for SwiGLU-Transformers through Layer-Wise Target Propagation
  di: Jantsch, Lasse Marten, et al.
  Pubblicazione: (2026)
• Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers
  di: Lau, Tim Tsz-Kit, et al.
  Pubblicazione: (2026)
• AudioMAE++: learning better masked audio representations with SwiGLU FFNs
  di: Yadav, Sarthak, et al.
  Pubblicazione: (2025)
• GLU Attention Improve Transformer
  di: Wang, Zehao
  Pubblicazione: (2025)
• Dependency-Aware Semi-Structured Sparsity of GLU Variants in Large Language Models
  di: Guo, Zhiyu, et al.
  Pubblicazione: (2024)