Cover Image

CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pham, Quang, Do, Giang, Nguyen, Huy, Nguyen, TrungTin, Liu, Chenghao, Sartipi, Mina, Nguyen, Binh T., Ramasamy, Savitha, Li, Xiaoli, Hoi, Steven, Ho, Nhat
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CompeteSMoE -- Statistically Guaranteed Mixture of Experts Training via Competition
by: Nguyen, Nam V., et al.
Published: (2025)

Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2023)

A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2023)

Dendrograms of Mixing Measures for Softmax-Gated Gaussian Mixture of Experts: Consistency without Model Sweeps
by: Hai, Do Tien, et al.
Published: (2025)

Model Selection for Gaussian-gated Gaussian Mixture of Experts Using Dendrograms of Mixing Measures
by: Thai, Tuan, et al.
Published: (2025)

Revisiting Incremental Stochastic Majorization-Minimization Algorithms with Applications to Mixture of Experts
by: Tran, TrungKhang, et al.
Published: (2026)

Sequence Transferability and Task Order Selection in Continual Learning
by: Nguyen, Thinh, et al.
Published: (2025)

Modifications of the BIC for order selection in finite mixture models
by: Nguyen, Hien Duy, et al.
Published: (2025)

Fast Model Selection and Stable Optimization for Softmax-Gated Multinomial-Logistic Mixture of Experts Models
by: Tran, TrungKhang, et al.
Published: (2026)

Risk Bounds for Mixture Density Estimation on Compact Domains via the $h$-Lifted Kullback--Leibler Divergence
by: Chong, Mark Chiu, et al.
Published: (2024)

On Parameter Estimation in Deviated Gaussian Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2024)

MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts
by: Teo, Rachel S. Y., et al.
Published: (2024)

A Unified Framework for Variable Selection in Model-Based Clustering with Missing Not at Random
by: Ho, Binh H., et al.
Published: (2025)

Mixture of Experts Meets Prompt-Based Continual Learning
by: Le, Minh, et al.
Published: (2024)

Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2023)

Statistical Advantages of Perturbing Cosine Router in Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2024)

Convergence Rates for Softmax Gating Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2025)

RepLoRA: Reparameterizing Low-Rank Adaptation via the Perspective of Mixture of Experts
by: Truong, Tuan, et al.
Published: (2025)

On Least Square Estimation in Softmax Gating Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2024)

Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts?
by: Nguyen, Huy, et al.
Published: (2024)

On the large-sample limits of some Bayesian model evaluation statistics
by: Nguyen, Hien Duy, et al.
Published: (2025)

Approximation rates for finite mixtures of location-scale models and fast least-squares estimators
by: Nguyen, Hien Duy, et al.
Published: (2025)

Rethinking Multinomial Logistic Mixture of Experts with Sigmoid Gating Function
by: Pham, Tuan Minh, et al.
Published: (2026)

Improving Minimax Estimation Rates for Contaminated Mixture of Multinomial Logistic Experts via Expert Heterogeneity
by: Yan, Fanqi, et al.
Published: (2026)

One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning
by: Le, Minh, et al.
Published: (2025)

Smart Camera Parking System With Auto Parking Spot Detection
by: Nguyen, Tuan T., et al.
Published: (2024)

Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2024)

On the Role of Discrete Representation in Sparse Mixture of Experts
by: Do, Giang, et al.
Published: (2024)

On Bayesian Softmax-Gated Mixture-of-Experts Models
by: Bariletto, Nicola, et al.
Published: (2026)

A Statistical Theory of Gated Attention through the Lens of Hierarchical Mixture of Experts
by: Nguyen, Viet, et al.
Published: (2026)

Quadratic Gating Mixture of Experts: Statistical Insights into Self-Attention
by: Akbarian, Pedram, et al.
Published: (2024)

Non-asymptotic oracle inequalities for the Lasso in high-dimensional mixture of experts
by: Nguyen, TrungTin, et al.
Published: (2020)

Selective Sinkhorn Routing for Improved Sparse Mixture of Experts
by: Nguyen, Duc Anh, et al.
Published: (2025)

Fourier-Attentive Representation Learning: A Fourier-Guided Framework for Few-Shot Generalization in Vision-Language Models
by: Pham, Hieu Dinh Trung, et al.
Published: (2025)

Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts
by: Yan, Fanqi, et al.
Published: (2024)

On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions
by: Nguyen, Huy, et al.
Published: (2024)

The Impact of Political Settings on Bank Competition: Evidence From US State‐Chartered Banks
by: Van Nguyen, et al.
Published: (2025)

Expert Merging in Sparse Mixture of Experts with Nash Bargaining
by: Nguyen, Dung V., et al.
Published: (2025)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
by: Nguyen-Nhat, Minh-Khoi, et al.
Published: (2025)

UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models
by: Tran, Van-Tuan, et al.
Published: (2026)