Copertina

Soft-to-Hard Routing in Sparse Mixture-of-Experts Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Rastegar, Reza
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence Probability
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Sparse Models, Sparse Safety: Unsafe Routes in Mixture-of-Experts LLMs
di: Jiang, Yukun, et al.
Pubblicazione: (2026)

From Sparse to Soft Mixtures of Experts
di: Puigcerver, Joan, et al.
Pubblicazione: (2023)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
di: Nguyen-Nhat, Minh-Khoi, et al.
Pubblicazione: (2025)

Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models
di: Liang, Jingcong, et al.
Pubblicazione: (2025)

RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMs
di: Xu, Zhiyuan, et al.
Pubblicazione: (2026)

Eradicating Negative Transfer in Multi-Physics Foundation Models via Sparse Mixture-of-Experts Routing
di: Sharma, Ellwil, et al.
Pubblicazione: (2026)

Routing-Free Mixture-of-Experts
di: Liu, Yilun, et al.
Pubblicazione: (2026)

Multilingual Routing in Mixture-of-Experts
di: Bandarkar, Lucas, et al.
Pubblicazione: (2025)

Mixture of Masters: Sparse Chess Language Models with Player Routing
di: Frisoni, Giacomo, et al.
Pubblicazione: (2026)

Towards Generalization-Oriented Models for Vehicle Routing Problems with Mixture-of-Experts
di: Miao, Changhao, et al.
Pubblicazione: (2026)

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
di: Zhan, Zheng, et al.
Pubblicazione: (2025)

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
di: Liu, Yilun, et al.
Pubblicazione: (2024)

Neural Inhibition Improves Dynamic Routing and Mixture of Experts
di: Zou, Will Y., et al.
Pubblicazione: (2025)

Probing Semantic Routing in Large Mixture-of-Expert Models
di: Olson, Matthew Lyle, et al.
Pubblicazione: (2025)

Exploring Expert Specialization through Unsupervised Training in Sparse Mixture of Experts
di: Nikolic, Strahinja, et al.
Pubblicazione: (2025)

ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts
di: Zhao, Heng, et al.
Pubblicazione: (2026)

Dense Backpropagation Improves Training for Sparse Mixture-of-Experts
di: Panda, Ashwinee, et al.
Pubblicazione: (2025)

Task-Conditioned Routing Signatures in Sparse Mixture-of-Experts Transformers
di: Avinash, Mynampati Sri Ranganadha
Pubblicazione: (2026)

Beyond Parameter Count: Implicit Bias in Soft Mixture of Experts
di: Chung, Youngseog, et al.
Pubblicazione: (2024)

RASA: Routing-Aware Safety Alignment for Mixture-of-Experts Models
di: Liang, Jiacheng, et al.
Pubblicazione: (2026)

Greedy Selection under Independent Increments: A Toy Model Analysis
di: Yang, Huitao
Pubblicazione: (2025)

MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts
di: Zhou, Jianan, et al.
Pubblicazione: (2024)

L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts
di: Yang, Minghao, et al.
Pubblicazione: (2026)

Variational Routing: A Scalable Bayesian Framework for Calibrated Mixture-of-Experts Transformers
di: Li, Albus Yizhuo, et al.
Pubblicazione: (2026)

Geometric Mixture-of-Experts with Curvature-Guided Adaptive Routing for Graph Representation Learning
di: Cao, Haifang, et al.
Pubblicazione: (2026)

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
di: Zhao, Yushu, et al.
Pubblicazione: (2025)

Omni-Router: Sharing Routing Decisions in Sparse Mixture-of-Experts for Speech Recognition
di: Gu, Zijin, et al.
Pubblicazione: (2025)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
di: Huang, Wei, et al.
Pubblicazione: (2025)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
di: He, Yifei, et al.
Pubblicazione: (2025)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
di: Liu, Yilun, et al.
Pubblicazione: (2025)

Robust Exploration in Directed Controller Synthesis via Reinforcement Learning with Soft Mixture-of-Experts
di: Ubukata, Toshihide, et al.
Pubblicazione: (2026)

$α$-TCAV: A Unified Framework for Testing with Concept Activation Vectors
di: Schnoor, Ekkehard, et al.
Pubblicazione: (2026)

Note on Martingale Theory and Applications
di: Zou, Xiandong
Pubblicazione: (2026)

On the Equivalence of Random Network Distillation, Deep Ensembles, and Bayesian Inference
di: Zanger, Moritz A., et al.
Pubblicazione: (2026)

Neural Network Parameter-optimization of Gaussian pmDAGs
di: Saremi, Mehrzad
Pubblicazione: (2023)

Deep Conditional Measure Quantization
di: Turinici, Gabriel
Pubblicazione: (2023)

Feature Learning Dynamics in Infinite-Depth Neural Networks
di: Yao, Zihan, et al.
Pubblicazione: (2025)

Causal Effect Identification in Heterogeneous Environments from Higher-Order Moments
di: Kivva, Yaroslav, et al.
Pubblicazione: (2025)

Explicit Density Approximation for Neural Implicit Samplers Using a Bernstein-Based Convex Divergence
di: de Frutos, José Manuel, et al.
Pubblicazione: (2025)

A Unified Theory of $θ$-Expectations
di: Qi, Qian
Pubblicazione: (2025)