Cover Image

CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Su, Zhenpeng, Wu, Xing, Lin, Zijia, Xiong, Yizhe, Lv, Minxuan, Ma, Guangyuan, Chen, Hui, Hu, Songlin, Ding, Guiguang
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts
by: Su, Zhenpeng, et al.
Published: (2024)

DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs
by: Lv, Minxuan, et al.
Published: (2025)

Finedeep: Mitigating Sparse Activation in Dense LLMs via Multi-Layer Fine-Grained Experts
by: Pan, Leiyu, et al.
Published: (2025)

MiLe Loss: a New Entropy-Weighed Loss for Mitigating the Bias of Learning Difficulties in Large Language Models
by: Su, Zhenpeng, et al.
Published: (2023)

UniAttn: Reducing Inference Costs via Softmax Unification for Post-Training LLMs
by: Xiong, Yizhe, et al.
Published: (2025)

Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems
by: Su, Zhenpeng, et al.
Published: (2023)

HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpus
by: Su, Zhenpeng, et al.
Published: (2023)

Drop your Decoder: Pre-training with Bag-of-Word Prediction for Dense Passage Retrieval
by: Ma, Guangyuan, et al.
Published: (2024)

Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal
by: Lian, Haoran, et al.
Published: (2024)

Cartesian-nj: Extending e3nn to Irreducible Cartesian Tensor Product and Contracion
by: Xu, Zemin, et al.
Published: (2025)

MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMs
by: Gu, Yupu, et al.
Published: (2026)

Multilingual Routing in Mixture-of-Experts
by: Bandarkar, Lucas, et al.
Published: (2025)

Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts
by: Jawahar, Ganesh, et al.
Published: (2023)

Temporal Scaling Law for Large Language Models
by: Xiong, Yizhe, et al.
Published: (2024)

Task-level Distributionally Robust Optimization for Large Language Model-based Dense Retrieval
by: Ma, Guangyuan, et al.
Published: (2024)

MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing
by: Go, Seokjin, et al.
Published: (2025)

Fast Quiet-STaR: Thinking Without Thought Tokens
by: Huang, Wei, et al.
Published: (2025)

LD-MoLE: Learnable Dynamic Routing for Mixture of LoRA Experts
by: Zhuang, Yuan, et al.
Published: (2025)

MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts
by: Xie, Zhitian, et al.
Published: (2024)

ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts
by: Zhao, Heng, et al.
Published: (2026)

AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models
by: Zeng, Zihao, et al.
Published: (2024)

Routing-Free Mixture-of-Experts
by: Liu, Yilun, et al.
Published: (2026)

CryptoMoE: Privacy-Preserving and Scalable Mixture of Experts Inference via Balanced Expert Routing
by: Zhou, Yifan, et al.
Published: (2025)

DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanism
by: Li, Dengchun, et al.
Published: (2025)

Lower General Position in Cartesian Products
by: Welton, Eartha Kruft, et al.
Published: (2024)

Projective Planar Cartesian Products of Graphs
by: Abell, Nicholas, et al.
Published: (2025)

LBPE: Long-token-first Tokenization to Improve Large Language Models
by: Lian, Haoran, et al.
Published: (2024)

Geometric Routing Enables Causal Expert Control in Mixture of Experts
by: Ternovtsii, Ivan, et al.
Published: (2026)

BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Experts
by: Wang, Qingyue, et al.
Published: (2025)

Omni-Router: Sharing Routing Decisions in Sparse Mixture-of-Experts for Speech Recognition
by: Gu, Zijin, et al.
Published: (2025)

BLR-MoE: Boosted Language-Routing Mixture of Experts for Domain-Robust Multilingual E2E ASR
by: Ma, Guodong, et al.
Published: (2025)

ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing
by: Wang, Ziteng, et al.
Published: (2024)

Maximum Score Routing For Mixture-of-Experts
by: Dong, Bowen, et al.
Published: (2025)

LightRetriever: A LLM-based Text Retrieval Architecture with Extremely Faster Query Inference
by: Ma, Guangyuan, et al.
Published: (2025)

LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing
by: Hao, Jiawei, et al.
Published: (2026)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
by: Takashiro, Shota, et al.
Published: (2026)

An EKR Theorem for the Cartesian Product of Complete Graphs
by: Joseph, Zaphenath
Published: (2025)

Learning Cartesian Product Graphs with Laplacian Constraints
by: Shi, Changhao, et al.
Published: (2024)

Optimal Adjacency Labels for Subgraphs of Cartesian Products
by: Esperet, Louis, et al.
Published: (2022)

MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts
by: Zhao, Yushu, et al.
Published: (2025)