Copertina

On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Nguyen, Huy, Han, Xing, Harris, Carl, Saria, Suchi, Ho, Nhat
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion
di: Han, Xing, et al.
Pubblicazione: (2024)

On Least Square Estimation in Softmax Gating Mixture of Experts
di: Nguyen, Huy, et al.
Pubblicazione: (2024)

Convergence Rates for Softmax Gating Mixture of Experts
di: Nguyen, Huy, et al.
Pubblicazione: (2025)

Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts
di: Nguyen, Huy, et al.
Pubblicazione: (2024)

On Bayesian Softmax-Gated Mixture-of-Experts Models
di: Bariletto, Nicola, et al.
Pubblicazione: (2026)

A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts
di: Nguyen, Huy, et al.
Pubblicazione: (2023)

Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts
di: Nguyen, Huy, et al.
Pubblicazione: (2023)

Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts?
di: Nguyen, Huy, et al.
Pubblicazione: (2024)

On Minimax Estimation of Parameters in Softmax-Contaminated Mixture of Experts
di: Yan, Fanqi, et al.
Pubblicazione: (2025)

Quadratic Gating Mixture of Experts: Statistical Insights into Self-Attention
di: Akbarian, Pedram, et al.
Pubblicazione: (2024)

On Parameter Estimation in Deviated Gaussian Mixture of Experts
di: Nguyen, Huy, et al.
Pubblicazione: (2024)

Rethinking Multinomial Logistic Mixture of Experts with Sigmoid Gating Function
di: Pham, Tuan Minh, et al.
Pubblicazione: (2026)

FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learning
di: Han, Xing, et al.
Pubblicazione: (2026)

A Statistical Theory of Gated Attention through the Lens of Hierarchical Mixture of Experts
di: Nguyen, Viet, et al.
Pubblicazione: (2026)

Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts
di: Nguyen, Huy, et al.
Pubblicazione: (2023)

Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts
di: Yan, Fanqi, et al.
Pubblicazione: (2024)

Massively Multimodal Foundation Models: A Framework for Capturing Interactions with Specialized Mixture-of-Experts
di: Han, Xing, et al.
Pubblicazione: (2025)

Dendrograms of Mixing Measures for Softmax-Gated Gaussian Mixture of Experts: Consistency without Model Sweeps
di: Hai, Do Tien, et al.
Pubblicazione: (2025)

Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspective
di: Yan, Fanqi, et al.
Pubblicazione: (2025)

Improving Minimax Estimation Rates for Contaminated Mixture of Multinomial Logistic Experts via Expert Heterogeneity
di: Yan, Fanqi, et al.
Pubblicazione: (2026)

Fast Model Selection and Stable Optimization for Softmax-Gated Multinomial-Logistic Mixture of Experts Models
di: Tran, TrungKhang, et al.
Pubblicazione: (2026)

Statistical Advantages of Perturbing Cosine Router in Mixture of Experts
di: Nguyen, Huy, et al.
Pubblicazione: (2024)

Mixture of Experts Meets Prompt-Based Continual Learning
di: Le, Minh, et al.
Pubblicazione: (2024)

RepLoRA: Reparameterizing Low-Rank Adaptation via the Perspective of Mixture of Experts
di: Truong, Tuan, et al.
Pubblicazione: (2025)

One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning
di: Le, Minh, et al.
Pubblicazione: (2025)

On DeepSeekMoE: Statistical Benefits of Shared Experts and Normalized Sigmoid Gating
di: Nguyen, Huy, et al.
Pubblicazione: (2025)

Gaussian Process-Gated Hierarchical Mixtures of Experts
di: Liu, Yuhao, et al.
Pubblicazione: (2023)

WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales
di: Prinster, Drew, et al.
Pubblicazione: (2025)

CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition
di: Pham, Quang, et al.
Pubblicazione: (2024)

On the Invariance and Generality of Neural Scaling Laws
di: Han, Xing, et al.
Pubblicazione: (2026)

Revisit Visual Prompt Tuning: The Expressiveness of Prompt Experts
di: Le, Minh, et al.
Pubblicazione: (2025)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
di: Nguyen-Nhat, Minh-Khoi, et al.
Pubblicazione: (2025)

Expert Merging in Sparse Mixture of Experts with Nash Bargaining
di: Nguyen, Dung V., et al.
Pubblicazione: (2025)

Model Selection for Gaussian-gated Gaussian Mixture of Experts Using Dendrograms of Mixing Measures
di: Thai, Tuan, et al.
Pubblicazione: (2025)

On Zero-Initialized Attention: Optimal Prompt and Gating Factor Estimation
di: Diep, Nghiem T., et al.
Pubblicazione: (2025)

Mixture-of-Experts for Distributed Edge Computing with Channel-Aware Gating Function
di: Song, Qiuchen, et al.
Pubblicazione: (2025)

Open-Set Domain Adaptation Under Background Distribution Shift: Challenges and A Provably Efficient Solution
di: Chaudhari, Shravan, et al.
Pubblicazione: (2025)

Optimal Transport Aggregation for Distributed Mixture-of-Experts
di: Chamroukhi, Faïcel, et al.
Pubblicazione: (2023)

MILM: Large Language Models for Multimodal Irregular Time Series with Informative Sampling
di: Chung, Hsing-Huan, et al.
Pubblicazione: (2026)

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
di: Zheng, Chen, et al.
Pubblicazione: (2025)