Cover Image

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhan, Zheng, Ren, Liliang, Wang, Shuohang, Liu, Liyuan, Liu, Yang, Gong, Yeyun, Wang, Yanzhi, Shen, Yelong
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy
by: Zhang, Rongzhi, et al.
Published: (2024)

Reinforcement Learning for Reasoning in Large Language Models with One Training Example
by: Wang, Yiping, et al.
Published: (2025)

BlackMamba: Mixture of Experts for State-Space Models
by: Anthony, Quentin, et al.
Published: (2024)

Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space
by: Zhang, Zhen, et al.
Published: (2025)

Adapting LLM Agents with Universal Feedback in Communication
by: Wang, Kuan, et al.
Published: (2023)

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
by: Pióro, Maciej, et al.
Published: (2024)

CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
by: Gou, Zhibin, et al.
Published: (2023)

Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning
by: Huang, Yiming, et al.
Published: (2024)

Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models
by: Sun, Jiashuo, et al.
Published: (2023)

Exploring Token Pruning in Vision State Space Models
by: Zhan, Zheng, et al.
Published: (2024)

Routing-Free Mixture-of-Experts
by: Liu, Yilun, et al.
Published: (2026)

Rethinking Language Model Scaling under Transferable Hypersphere Optimization
by: Ren, Liliang, et al.
Published: (2026)

Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models
by: Liang, Jingcong, et al.
Published: (2025)

RASA: Routing-Aware Safety Alignment for Mixture-of-Experts Models
by: Liang, Jiacheng, et al.
Published: (2026)

Multilingual Routing in Mixture-of-Experts
by: Bandarkar, Lucas, et al.
Published: (2025)

ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
by: Gou, Zhibin, et al.
Published: (2023)

Mixture of Heterogeneous Grouped Experts for Language Modeling
by: Ma, Zhicheng, et al.
Published: (2026)

Geometric Routing Enables Causal Expert Control in Mixture of Experts
by: Ternovtsii, Ivan, et al.
Published: (2026)

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
by: Liu, Yilun, et al.
Published: (2024)

L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts
by: Yang, Minghao, et al.
Published: (2026)

Expert Race: A Flexible Routing Strategy for Scaling Diffusion Transformer with Mixture of Experts
by: Yuan, Yike, et al.
Published: (2025)

Multi-Task Vehicle Routing Solver via Mixture of Specialized Experts under State-Decomposable MDP
by: Pan, Yuxin, et al.
Published: (2025)

Cost-Effective Proxy Reward Model Construction with On-Policy and Active Learning
by: Chen, Yifang, et al.
Published: (2024)

Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
by: Ren, Liliang, et al.
Published: (2024)

AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models
by: Zeng, Zihao, et al.
Published: (2024)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
by: He, Yifei, et al.
Published: (2025)

Routing-Aligned Fine-Tuning for Multilingual Downstream Tasks in Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

Soft-to-Hard Routing in Sparse Mixture-of-Experts Models
by: Rastegar, Reza
Published: (2026)

RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMs
by: Xu, Zhiyuan, et al.
Published: (2026)

MambaFormer: Token-Level Guided Routing Mixture-of-Experts for Accurate and Efficient Clinical Assistance
by: Khan, Hamad, et al.
Published: (2026)

StyleMamba : State Space Model for Efficient Text-driven Image Style Transfer
by: Wang, Zijia, et al.
Published: (2024)

Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsity
by: Liang, Weixin, et al.
Published: (2025)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
by: Xie, Yanyue, et al.
Published: (2024)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
by: Gao, Yuting, et al.
Published: (2025)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
by: Chu, Kexin, et al.
Published: (2025)

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
by: Shin, Haebin, et al.
Published: (2025)

Equifinality in Mixture of Experts: Routing Topology Does Not Determine Language Modeling Quality
by: Ternovtsii, Ivan, et al.
Published: (2026)

Probing Semantic Routing in Large Mixture-of-Expert Models
by: Olson, Matthew Lyle, et al.
Published: (2025)

Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts
by: Xu, Haolei, et al.
Published: (2026)

Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling
by: Shin, Haebin, et al.
Published: (2025)