Image de couverture de livre

Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yan, Fanqi, Nguyen, Huy, Le, Dung, Akbarian, Pedram, Ho, Nhat
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

On Minimax Estimation of Parameters in Softmax-Contaminated Mixture of Experts
par: Yan, Fanqi, et autres
Publié: (2025)

Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspective
par: Yan, Fanqi, et autres
Publié: (2025)

Improving Minimax Estimation Rates for Contaminated Mixture of Multinomial Logistic Experts via Expert Heterogeneity
par: Yan, Fanqi, et autres
Publié: (2026)

Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts
par: Nguyen, Huy, et autres
Publié: (2023)

Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts?
par: Nguyen, Huy, et autres
Publié: (2024)

Quadratic Gating Mixture of Experts: Statistical Insights into Self-Attention
par: Akbarian, Pedram, et autres
Publié: (2024)

A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts
par: Nguyen, Huy, et autres
Publié: (2023)

Statistical Advantages of Perturbing Cosine Router in Mixture of Experts
par: Nguyen, Huy, et autres
Publié: (2024)

On Parameter Estimation in Deviated Gaussian Mixture of Experts
par: Nguyen, Huy, et autres
Publié: (2024)

Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts
par: Nguyen, Huy, et autres
Publié: (2023)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
par: Nguyen-Nhat, Minh-Khoi, et autres
Publié: (2025)

On Least Square Estimation in Softmax Gating Mixture of Experts
par: Nguyen, Huy, et autres
Publié: (2024)

On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions
par: Nguyen, Huy, et autres
Publié: (2024)

Convergence Rates for Softmax Gating Mixture of Experts
par: Nguyen, Huy, et autres
Publié: (2025)

Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts
par: Nguyen, Huy, et autres
Publié: (2024)

MP-MoE: Matrix Profile-Guided Mixture of Experts for Precipitation Forecasting
par: Tran, Huyen Ngoc, et autres
Publié: (2026)

Mixture of Experts Meets Prompt-Based Continual Learning
par: Le, Minh, et autres
Publié: (2024)

Optimal Transport Aggregation for Distributed Mixture-of-Experts
par: Chamroukhi, Faïcel, et autres
Publié: (2023)

RepLoRA: Reparameterizing Low-Rank Adaptation via the Perspective of Mixture of Experts
par: Truong, Tuan, et autres
Publié: (2025)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
par: Yan, Jiaming, et autres
Publié: (2025)

Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Experts
par: Nguyen, Xuan-Phi, et autres
Publié: (2026)

Mixtures of Experts Unlock Parameter Scaling for Deep RL
par: Obando-Ceron, Johan, et autres
Publié: (2024)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
par: Jin, Peng, et autres
Publié: (2024)

Speculating Experts Accelerates Inference for Mixture-of-Experts
par: Madan, Vivan, et autres
Publié: (2026)

Beyond Parameter Count: Implicit Bias in Soft Mixture of Experts
par: Chung, Youngseog, et autres
Publié: (2024)

FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion
par: Han, Xing, et autres
Publié: (2024)

On Bayesian Softmax-Gated Mixture-of-Experts Models
par: Bariletto, Nicola, et autres
Publié: (2026)

Mixture of Raytraced Experts
par: Perin, Andrea, et autres
Publié: (2025)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
par: Gao, Yuting, et autres
Publié: (2025)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
par: He, Yifei, et autres
Publié: (2025)

One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning
par: Le, Minh, et autres
Publié: (2025)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
par: Huang, Wei, et autres
Publié: (2025)

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
par: Liu, Yilun, et autres
Publié: (2024)

On the Spatial Structure of Mixture-of-Experts in Transformers
par: Bershatsky, Daniel, et autres
Publié: (2025)

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
par: Dwivedi, Chaitanya, et autres
Publié: (2026)

Rethinking Multinomial Logistic Mixture of Experts with Sigmoid Gating Function
par: Pham, Tuan Minh, et autres
Publié: (2026)

Routing-Free Mixture-of-Experts
par: Liu, Yilun, et autres
Publié: (2026)

Mixture of Diverse Size Experts
par: Sun, Manxi, et autres
Publié: (2024)

Mixture of A Million Experts
par: He, Xu Owen
Publié: (2024)

Sparsity and Superposition in Mixture of Experts
par: Chaudhari, Marmik, et autres
Publié: (2025)