Copertina

Expert Divergence Learning for MoE-based Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Jiaang, Chen, Haibin, Liu, Langming, Yuan, Yujin, Wang, Yadao, Zhang, Yizhen, Yu, Chengting, Tong, Xin, Zhang, Weidong, Liu, Shilei, Su, Wenbo, Zheng, Bo
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

ECKGBench: Benchmarking Large Language Models in E-commerce Leveraging Knowledge Graph
di: Liu, Langming, et al.
Pubblicazione: (2025)

PretrainRL: Alleviating Factuality Hallucination of Large Language Models at the Beginning
di: Liu, Langming, et al.
Pubblicazione: (2026)

Data Distribution Matters: A Data-Centric Perspective on Context Compression for Large Language Model
di: Lv, Kangtao, et al.
Pubblicazione: (2026)

How to inject knowledge efficiently? Knowledge Infusion Scaling Law for Pre-training Large Language Models
di: Lv, Kangtao, et al.
Pubblicazione: (2025)

PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction
di: Zhao, Runsong, et al.
Pubblicazione: (2026)

CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling
di: Zhao, Runsong, et al.
Pubblicazione: (2026)

ChineseEcomQA: A Scalable E-commerce Concept Evaluation Benchmark for Large Language Models
di: Chen, Haibin, et al.
Pubblicazione: (2025)

MeSH: Memory-as-State-Highways for Recursive Transformers
di: Yu, Chengting, et al.
Pubblicazione: (2025)

Unlocking Scaling Law in Industrial Recommendation Systems with a Three-step Paradigm based Large User Model
di: Yan, Bencheng, et al.
Pubblicazione: (2025)

Read As Human: Compressing Context via Parallelizable Close Reading and Skimming
di: Tang, Jiwei, et al.
Pubblicazione: (2026)

UQABench: Evaluating User Embedding for Prompting LLMs in Personalized Question Answering
di: Liu, Langming, et al.
Pubblicazione: (2025)

SpiralFormer: Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursion
di: Yu, Chengting, et al.
Pubblicazione: (2026)

COMI: Coarse-to-fine Context Compression via Marginal Information Gain
di: Tang, Jiwei, et al.
Pubblicazione: (2026)

Fair-MoE: Fairness-Oriented Mixture of Experts in Vision-Language Models
di: Wang, Peiran, et al.
Pubblicazione: (2025)

Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models
di: Jiang, Songtao, et al.
Pubblicazione: (2024)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
di: Jin, Peng, et al.
Pubblicazione: (2024)

CoMoE: Collaborative Optimization of Expert Aggregation and Offloading for MoE-based LLMs at Edge
di: Li, Muqing, et al.
Pubblicazione: (2025)

Steer-MoE: Efficient Audio-Language Alignment with a Mixture-of-Experts Steering Module
di: Feng, Ruitao, et al.
Pubblicazione: (2025)

EvoESAP: Non-Uniform Expert Pruning for Sparse MoE
di: Liu, Zongfang, et al.
Pubblicazione: (2026)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
di: Ma, Songkai, et al.
Pubblicazione: (2025)

MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
di: Chen, Xiaodong, et al.
Pubblicazione: (2025)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
di: Jin, Chao, et al.
Pubblicazione: (2025)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
di: Qian, Yulei, et al.
Pubblicazione: (2024)

BrainNet-MoE: Brain-Inspired Mixture-of-Experts Learning for Neurological Disease Identification
di: Zhang, Jing, et al.
Pubblicazione: (2025)

Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs
di: Zhou, Xin, et al.
Pubblicazione: (2024)

OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
di: Wang, Liujianfu, et al.
Pubblicazione: (2025)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
di: Xia, Xinfeng, et al.
Pubblicazione: (2025)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
di: Wu, Haoyuan, et al.
Pubblicazione: (2025)

Polysemantic Experts, Monosemantic Paths: Routing as Control in MoEs
di: Ye, Charles, et al.
Pubblicazione: (2026)

MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
di: Lin, Bin, et al.
Pubblicazione: (2024)

MoE-DisCo:Low Economy Cost Training Mixture-of-Experts Models
di: Ye, Xin, et al.
Pubblicazione: (2026)

MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
di: Miao, Ruijie, et al.
Pubblicazione: (2025)

CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling
di: Zhang, Jihai, et al.
Pubblicazione: (2024)

YOCO++: Enhancing YOCO with KV Residual Connections for Efficient LLM Inference
di: Wu, You, et al.
Pubblicazione: (2026)

SD-MoE: Spectral Decomposition for Effective Expert Specialization
di: Huang, Ruijun, et al.
Pubblicazione: (2026)

Routing Matters in MoE: Scaling Diffusion Transformers with Explicit Routing Guidance
di: Wei, Yujie, et al.
Pubblicazione: (2025)

Advancing Expert Specialization for Better MoE
di: Guo, Hongcan, et al.
Pubblicazione: (2025)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
di: Xie, Yanyue, et al.
Pubblicazione: (2024)

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
di: Shi, Jingze, et al.
Pubblicazione: (2026)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
di: Liu, Baihui, et al.
Pubblicazione: (2026)