Cover Image

Yuan 2.0-M32: Mixture of Experts with Attention Router

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wu, Shaohua, Luo, Jiangang, Chen, Xi, Li, Lingjun, Zhao, Xudong, Yu, Tong, Wang, Chao, Wang, Yue, Wang, Fei, Qiao, Weixu, He, Houbo, Zhang, Zeru, Sun, Zeyu, Mao, Junxiong, Shen, Chong
Format:	Preprint
Published:	2024
Subjects:	Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Yuan-TecSwin: A text conditioned Diffusion model with Swin-transformer blocks
by: Wu, Shaohua, et al.
Published: (2025)

Layerwise Recurrent Router for Mixture-of-Experts
by: Qiu, Zihan, et al.
Published: (2024)

Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling
by: Ran, Junfeng, et al.
Published: (2025)

Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss
by: Lv, Ang, et al.
Published: (2025)

Mixture of Routers
by: Zhang, Jia-Chen, et al.
Published: (2025)

Routers in Vision Mixture of Experts: An Empirical Study
by: Liu, Tianlin, et al.
Published: (2024)

RouterKT: Mixture-of-Experts for Knowledge Tracing
by: Liao, Han, et al.
Published: (2025)

Graph Mixture of Experts and Memory-augmented Routers for Multivariate Time Series Anomaly Detection
by: Huang, Xiaoyu, et al.
Published: (2024)

Mixture-of-Schedulers: An Adaptive Scheduling Agent as a Learned Router for Expert Policies
by: Wang, Xinbo, et al.
Published: (2025)

Read-ME: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design
by: Cai, Ruisi, et al.
Published: (2024)

Load Balancing Mixture of Experts with Similarity Preserving Routers
by: Omi, Nabil, et al.
Published: (2025)

Statistical Advantages of Perturbing Cosine Router in Mixture of Experts
by: Nguyen, Huy, et al.
Published: (2024)

Part-Of-Speech Sensitivity of Routers in Mixture of Experts Models
by: Antoine, Elie, et al.
Published: (2024)

Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
by: Ahrac, Sagi, et al.
Published: (2026)

Long-Tailed Distribution-Aware Router For Mixture-of-Experts in Large Vision-Language Model
by: Cai, Chaoxiang, et al.
Published: (2025)

Evolutionary Router Feature Generation for Zero-Shot Graph Anomaly Detection with Mixture-of-Experts
by: Jiang, Haiyang, et al.
Published: (2026)

Yuan3.0 Flash: An Open Multimodal Large Language Model for Enterprise Applications
by: ai, YuanLab., et al.
Published: (2026)

RouterRetriever: Routing over a Mixture of Expert Embedding Models
by: Lee, Hyunji, et al.
Published: (2024)

Yuan3.0 Ultra: A Trillion-Parameter Enterprise-Oriented MoE LLM
by: ai, YuanLab., et al.
Published: (2026)

Enhanced Small Target Detection via Multi-Modal Fusion and Attention Mechanisms: A YOLOv5 Approach
by: Ma, Xiaoxiao, et al.
Published: (2025)

Omni-Router: Sharing Routing Decisions in Sparse Mixture-of-Experts for Speech Recognition
by: Gu, Zijin, et al.
Published: (2025)

Optimus-3: Dual-Router Aligned Mixture-of-Experts Agent with Dual-Granularity Reasoning-Aware Policy Optimization
by: Li, Zaijing, et al.
Published: (2025)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
by: Xie, Yanyue, et al.
Published: (2024)

Temporally Extended Mixture-of-Experts Models
by: Shen, Zeyu, et al.
Published: (2026)

GRIP: Algorithm-Agnostic Machine Unlearning for Mixture-of-Experts via Geometric Router Constraints
by: Zhu, Andy, et al.
Published: (2026)

A Minimal Bifurcation Model of Load Imbalance in a Softmax Mixture-of-Experts Router
by: Kiselev, O. M.
Published: (2026)

FLEx: Personalized Federated Learning for Mixture-of-Experts LLMs via Expert Grafting
by: Liu, Fan, et al.
Published: (2025)

AFter: Attention-based Fusion Router for RGBT Tracking
by: Lu, Andong, et al.
Published: (2024)

A Closer Look into Mixture-of-Experts in Large Language Models
by: Lo, Ka Man, et al.
Published: (2024)

ViTE: Virtual Graph Trajectory Expert Router for Pedestrian Trajectory Prediction
by: Li, Ruochen, et al.
Published: (2025)

DIME-Net: A Dual-Illumination Adaptive Enhancement Network Based on Retinex and Mixture-of-Experts
by: Wang, Ziang, et al.
Published: (2025)

CometNet: Contextual Motif-guided Long-term Time Series Forecasting
by: Wang, Weixu, et al.
Published: (2025)

MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
by: Zhu, Ruidong, et al.
Published: (2025)

Yuan Yang
Published: (2024)

Opportunistic Expert Activation: Batch-Aware Expert Routing for Faster Decode Without Retraining
by: Oncescu, Costin-Andrei, et al.
Published: (2025)

Mixture of Lookup Key-Value Experts
by: Wang, Zongcheng
Published: (2025)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
by: Liu, Baihui, et al.
Published: (2026)

Hierarchically Decoupled Mixture-of-Experts for Robust Traffic Sign Recognition in Complex Driving Scenarios
by: Wang, Mingxiao, et al.
Published: (2026)

Imitation Learning from Observations: An Autoregressive Mixture of Experts Approach
by: Wang, Renzi, et al.
Published: (2024)

Learned Inertial Odometry for Cycling Based on Mixture of Experts Algorithm
by: Qiao, Hao, et al.
Published: (2025)