Copertina

Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Su, Guinan, Yang, Yanwu, Shen, Li, Yin, Lu, Liu, Shiwei, Geiping, Jonas
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

GPTailor: Large Language Model Pruning Through Layer Cutting and Stitching
di: Su, Guinan, et al.
Pubblicazione: (2025)

Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs
di: Su, Guinan, et al.
Pubblicazione: (2026)

Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs
di: Li, Xueyan, et al.
Pubblicazione: (2025)

Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Models
di: Geiping, Jonas, et al.
Pubblicazione: (2025)

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
di: Wang, Zihan, et al.
Pubblicazione: (2025)

MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts
di: Zhao, Yushu, et al.
Pubblicazione: (2025)

MedSAMix: A Training-Free Model Merging Approach for Medical Image Segmentation
di: Yang, Yanwu, et al.
Pubblicazione: (2025)

Training Matryoshka Mixture-of-Experts for Elastic Inference-Time Expert Utilization
di: Wang, Yaoxiang, et al.
Pubblicazione: (2025)

Unveiling Super Experts in Mixture-of-Experts Large Language Models
di: Su, Zunhai, et al.
Pubblicazione: (2025)

Mixture of Neuron Experts
di: Cheng, Runxi, et al.
Pubblicazione: (2025)

Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMs
di: Bai, Jun, et al.
Pubblicazione: (2025)

Fine, I'll Merge It Myself: A Multi-Fidelity Framework for Automated Model Merging
di: Su, Guinan, et al.
Pubblicazione: (2025)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
di: Liu, Yilun, et al.
Pubblicazione: (2025)

ReXMoE: Reusing Experts with Minimal Overhead in Mixture-of-Experts
di: Tan, Zheyue, et al.
Pubblicazione: (2025)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
di: Lu, Xudong, et al.
Pubblicazione: (2024)

Mixture of Lookup Experts
di: Jie, Shibo, et al.
Pubblicazione: (2025)

Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Experts
di: Zhang, Xue, et al.
Pubblicazione: (2025)

Cluster-Driven Expert Pruning for Mixture-of-Experts Large Language Models
di: Guo, Hongcheng, et al.
Pubblicazione: (2025)

ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
di: He, Xin, et al.
Pubblicazione: (2024)

Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts
di: Xu, Haolei, et al.
Pubblicazione: (2026)

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
di: Liu, Yang, et al.
Pubblicazione: (2026)

Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling
di: Ran, Junfeng, et al.
Pubblicazione: (2025)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
di: Huang, Wei, et al.
Pubblicazione: (2024)

Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment
di: Liu, Zhili, et al.
Pubblicazione: (2024)

DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning
di: Bai, Sikai, et al.
Pubblicazione: (2025)

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
di: Zheng, Chen, et al.
Pubblicazione: (2025)

SciDFM: A Large Language Model with Mixture-of-Experts for Science
di: Sun, Liangtai, et al.
Pubblicazione: (2024)

Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
di: He, Haoze, et al.
Pubblicazione: (2026)

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
di: Zhang, Zeliang, et al.
Pubblicazione: (2024)

PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning
di: Li, Zongqian, et al.
Pubblicazione: (2025)

SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
di: Zhuang, Haomin, et al.
Pubblicazione: (2024)

Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning
di: Zhang, Yichi, et al.
Pubblicazione: (2024)

Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer
di: Liu, Boan, et al.
Pubblicazione: (2023)

Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs
di: Zhou, Yixiao, et al.
Pubblicazione: (2025)

Mixture of Heterogeneous Grouped Experts for Language Modeling
di: Ma, Zhicheng, et al.
Pubblicazione: (2026)

LLaVA-CMoE: Towards Continual Mixture of Experts for Large Vision-Language Models
di: Zhao, Hengyuan, et al.
Pubblicazione: (2025)

Mixtures of SubExperts for Large Language Continual Learning
di: Kang, Haeyong
Pubblicazione: (2025)

Multilingual Routing in Mixture-of-Experts
di: Bandarkar, Lucas, et al.
Pubblicazione: (2025)

Improving the Downstream Performance of Mixture-of-Experts Transformers via Weak Vanilla Transformers
di: Lu, Xin, et al.
Pubblicazione: (2024)

Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts
di: Cai, Weilin, et al.
Pubblicazione: (2024)