Cover Image

Part-Of-Speech Sensitivity of Routers in Mixture of Experts Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Antoine, Elie, Béchet, Frédéric, Langlais, Philippe
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A linguistically-motivated evaluation methodology for unraveling model's abilities in reading comprehension tasks
by: Antoine, Elie, et al.
Published: (2025)

Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling
by: Ran, Junfeng, et al.
Published: (2025)

Layerwise Recurrent Router for Mixture-of-Experts
by: Qiu, Zihan, et al.
Published: (2024)

On Evaluation Protocols for Data Augmentation in a Limited Data Scenario
by: Piedboeuf, Frédéric, et al.
Published: (2024)

Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss
by: Lv, Ang, et al.
Published: (2025)

Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
by: Ahrac, Sagi, et al.
Published: (2026)

Omni-Router: Sharing Routing Decisions in Sparse Mixture-of-Experts for Speech Recognition
by: Gu, Zijin, et al.
Published: (2025)

Mixture of Routers
by: Zhang, Jia-Chen, et al.
Published: (2025)

$\textit{BenchIE}^{FL}$ : A Manually Re-Annotated Fact-Based Open Information Extraction Benchmark
by: Lamarche, Fabrice, et al.
Published: (2024)

Yuan 2.0-M32: Mixture of Experts with Attention Router
by: Wu, Shaohua, et al.
Published: (2024)

Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering
by: Kobeissi, Amine, et al.
Published: (2026)

Read-ME: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design
by: Cai, Ruisi, et al.
Published: (2024)

On the importance of Data Scale in Pretraining Arabic Language Models
by: Ghaddar, Abbas, et al.
Published: (2024)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
by: Xie, Yanyue, et al.
Published: (2024)

Increasing faithfulness in human-human dialog summarization with Spoken Language Understanding tasks
by: Akani, Eunice, et al.
Published: (2024)

WikiFactDiff: A Large, Realistic, and Temporally Adaptable Dataset for Atomic Factual Knowledge Update in Causal Language Models
by: Khodja, Hichem Ammar, et al.
Published: (2024)

Robust Audiovisual Speech Recognition Models with Mixture-of-Experts
by: Wu, Yihan, et al.
Published: (2024)

Glider: Global and Local Instruction-Driven Expert Router
by: Li, Pingzhi, et al.
Published: (2024)

MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Experts
by: Yu, Haofei, et al.
Published: (2023)

Factual Knowledge in Language Models: Robustness and Anomalies under Simple Temporal Context Variations
by: Khodja, Hichem Ammar, et al.
Published: (2025)

Performance Characterization of Expert Router for Scalable LLM Inference
by: Pichlmeier, Josef, et al.
Published: (2024)

Mixture-of-Experts with Intermediate CTC Supervision for Accented Speech Recognition
by: Lee, Wonjun, et al.
Published: (2026)

CareMedEval dataset: Evaluating Critical Appraisal and Reasoning in the Biomedical Field
by: Bonzi, Doria, et al.
Published: (2025)

EUROPA: A Legal Multilingual Keyphrase Generation Dataset
by: Salaün, Olivier, et al.
Published: (2024)

Unveiling Super Experts in Mixture-of-Experts Large Language Models
by: Su, Zunhai, et al.
Published: (2025)

Mixture of LoRA Experts for Low-Resourced Multi-Accent Automatic Speech Recognition
by: Bagat, Raphaël, et al.
Published: (2025)

Cluster-Driven Expert Pruning for Mixture-of-Experts Large Language Models
by: Guo, Hongcheng, et al.
Published: (2025)

ReGLA: Refining Gated Linear Attention
by: Lu, Peng, et al.
Published: (2025)

Mixture of Neuron Experts
by: Cheng, Runxi, et al.
Published: (2025)

MoDEM: Mixture of Domain Expert Models
by: Simonds, Toby, et al.
Published: (2024)

CP-Router: An Uncertainty-Aware Router Between LLM and LRM
by: Su, Jiayuan, et al.
Published: (2025)

Style Mixture of Experts for Expressive Text-To-Speech Synthesis
by: Jawaid, Ahad, et al.
Published: (2024)

Enhancing Code-Switching Speech Recognition with LID-Based Collaborative Mixture of Experts Model
by: Huang, Hukai, et al.
Published: (2024)

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
by: Wang, Zihan, et al.
Published: (2025)

EvoMoE: Expert Evolution in Mixture of Experts for Multimodal Large Language Models
by: Jing, Linglin, et al.
Published: (2025)

Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models
by: Belenki, Lior, et al.
Published: (2025)

MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts
by: Zhao, Yushu, et al.
Published: (2025)

LABO: Towards Learning Optimal Label Regularization via Bi-level Optimization
by: Lu, Peng, et al.
Published: (2023)

Mixture of Lookup Experts
by: Jie, Shibo, et al.
Published: (2025)

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
by: Dai, Damai, et al.
Published: (2024)