Copertina

Dynamic Mixture of Experts Against Severe Distribution Shifts

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Kim, Donghu
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
di: Dwivedi, Chaitanya, et al.
Pubblicazione: (2026)

Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learning
di: Bhope, Rahul Atul, et al.
Pubblicazione: (2025)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
di: Chu, Kexin, et al.
Pubblicazione: (2025)

Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts
di: Li, Cheng, et al.
Pubblicazione: (2025)

Geometric Regularization in Mixture-of-Experts: The Disconnect Between Weights and Activations
di: Kim, Hyunjun
Pubblicazione: (2026)

Neural Inhibition Improves Dynamic Routing and Mixture of Experts
di: Zou, Will Y., et al.
Pubblicazione: (2025)

Do's and Don'ts: Learning Desirable Skills with Instruction Videos
di: Kim, Hyunseung, et al.
Pubblicazione: (2024)

Mixture of Raytraced Experts
di: Perin, Andrea, et al.
Pubblicazione: (2025)

Mixture of Experts in a Mixture of RL settings
di: Willi, Timon, et al.
Pubblicazione: (2024)

Optimal Expert-Attention Allocation in Mixture-of-Experts: A Scalable Law for Dynamic Model Design
di: Li, Junzhuo, et al.
Pubblicazione: (2026)

Speculating Experts Accelerates Inference for Mixture-of-Experts
di: Madan, Vivan, et al.
Pubblicazione: (2026)

FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoff
di: Han, Isaac, et al.
Pubblicazione: (2026)

Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
di: Guo, Yongxin, et al.
Pubblicazione: (2024)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
di: Huang, Wei, et al.
Pubblicazione: (2025)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
di: He, Yifei, et al.
Pubblicazione: (2025)

Sparsity and Superposition in Mixture of Experts
di: Chaudhari, Marmik, et al.
Pubblicazione: (2025)

Mixture of Diverse Size Experts
di: Sun, Manxi, et al.
Pubblicazione: (2024)

Mixture of Concept Bottleneck Experts
di: De Santis, Francesco, et al.
Pubblicazione: (2026)

Mixture of A Million Experts
di: He, Xu Owen
Pubblicazione: (2024)

Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networks
di: Lee, Hojoon, et al.
Pubblicazione: (2024)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
di: Gao, Yuting, et al.
Pubblicazione: (2025)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
di: Kim, Junhyuck, et al.
Pubblicazione: (2026)

Adaptive Individual Uncertainty under Out-Of-Distribution Shift with Expert-Routed Conformal Prediction
di: Badkul, Amitesh, et al.
Pubblicazione: (2025)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
di: Yan, Jiaming, et al.
Pubblicazione: (2025)

Dynamic Experts Search: Enhancing Reasoning in Mixture-of-Experts LLMs at Test Time
di: Han, Yixuan, et al.
Pubblicazione: (2025)

Mixture of Experts in Large Language Models
di: Zhang, Danyang, et al.
Pubblicazione: (2025)

Graph Knowledge Distillation to Mixture of Experts
di: Rumiantsev, Pavel, et al.
Pubblicazione: (2024)

Mixture of Weak & Strong Experts on Graphs
di: Zeng, Hanqing, et al.
Pubblicazione: (2023)

Theory on Mixture-of-Experts in Continual Learning
di: Li, Hongbo, et al.
Pubblicazione: (2024)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
di: Nguyen-Nhat, Minh-Khoi, et al.
Pubblicazione: (2025)

Exploring Expert Specialization through Unsupervised Training in Sparse Mixture of Experts
di: Nikolic, Strahinja, et al.
Pubblicazione: (2025)

UniPool: A Globally Shared Expert Pool for Mixture-of-Experts
di: Huang, Minbin, et al.
Pubblicazione: (2026)

Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts
di: Yan, Fanqi, et al.
Pubblicazione: (2024)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
di: Jin, Peng, et al.
Pubblicazione: (2024)

Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Experts
di: Nguyen, Xuan-Phi, et al.
Pubblicazione: (2026)

Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
di: Park, Jongseok, et al.
Pubblicazione: (2026)

Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Models
di: Kim, Gyeongman, et al.
Pubblicazione: (2025)

DynaMoE: Dynamic Token-Level Expert Activation with Layer-Wise Adaptive Capacity for Mixture-of-Experts Neural Networks
di: Gülmez, Gökdeniz
Pubblicazione: (2026)

Sufficient Invariant Learning for Distribution Shift
di: Kim, Taero, et al.
Pubblicazione: (2022)

Optimal Transport Aggregation for Distributed Mixture-of-Experts
di: Chamroukhi, Faïcel, et al.
Pubblicazione: (2023)