Quadratic Gating Mixture of Experts: Statistical Insights into Self-Attention

Documenti analoghi

• Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts
  di: Nguyen, Huy, et al.
  Pubblicazione: (2023)
• Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts?
  di: Nguyen, Huy, et al.
  Pubblicazione: (2024)
• A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts
  di: Nguyen, Huy, et al.
  Pubblicazione: (2023)
• Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspective
  di: Yan, Fanqi, et al.
  Pubblicazione: (2025)
• Statistical Advantages of Perturbing Cosine Router in Mixture of Experts
  di: Nguyen, Huy, et al.
  Pubblicazione: (2024)