Copertina

Mixture of Experts in Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Danyang, Song, Junhao, Bi, Ziqian, Song, Xinyuan, Yuan, Yingfang, Wang, Tianyang, Yeong, Joe, Hao, Junfeng
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

47B Mixture-of-Experts Beats 671B Dense Models on Chinese Medical Examinations
di: Tseng, Chiung-Yi, et al.
Pubblicazione: (2025)

Achieving Trustworthy Real-Time Decision Support Systems with Low-Latency Interpretable AI Models
di: Deng, Zechun, et al.
Pubblicazione: (2025)

Is GPT-OSS All You Need? Benchmarking Large Language Models for Financial Intelligence and the Surprising Efficiency Paradox
di: Bi, Ziqian, et al.
Pubblicazione: (2025)

CoT-X: An Adaptive Framework for Cross-Model Chain-of-Thought Transfer and Optimization
di: Bi, Ziqian, et al.
Pubblicazione: (2025)

Jailbreaking and Mitigation of Vulnerabilities in Large Language Models
di: Peng, Benji, et al.
Pubblicazione: (2024)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
di: Huang, Wei, et al.
Pubblicazione: (2025)

EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
di: Chen, Yuanteng, et al.
Pubblicazione: (2025)

Unveiling Hidden Collaboration within Mixture-of-Experts in Large Language Models
di: Tang, Yuanbo, et al.
Pubblicazione: (2025)

T-REX: Mixture-of-Rank-One-Experts with Semantic-aware Intuition for Multi-task Large Language Model Finetuning
di: Zhang, Rongyu, et al.
Pubblicazione: (2024)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
di: Gao, Yuting, et al.
Pubblicazione: (2025)

Upcycling Large Language Models into Mixture of Experts
di: He, Ethan, et al.
Pubblicazione: (2024)

MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance
di: Hu, Xing, et al.
Pubblicazione: (2025)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
di: Xie, Yanyue, et al.
Pubblicazione: (2024)

HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts
di: He, Neil, et al.
Pubblicazione: (2025)

The Curse of Depth in Large Language Models
di: Sun, Wenfang, et al.
Pubblicazione: (2025)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
di: Lu, Xudong, et al.
Pubblicazione: (2024)

Deep Learning and Machine Learning, Advancing Big Data Analytics and Management: Tensorflow Pretrained Models
di: Chen, Keyu, et al.
Pubblicazione: (2024)

Manifold-Aware Temporal Domain Generalization for Large Language Models
di: Yao, Yiheng, et al.
Pubblicazione: (2026)

Mosaic Pruning: A Hierarchical Framework for Generalizable Pruning of Mixture-of-Experts Models
di: Hu, Wentao, et al.
Pubblicazione: (2025)

AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting
di: Wang, Rui, et al.
Pubblicazione: (2026)

WDMoE: Wireless Distributed Large Language Models with Mixture of Experts
di: Xue, Nan, et al.
Pubblicazione: (2024)

FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models
di: Zhao, Zhongyu, et al.
Pubblicazione: (2024)

MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
di: Liu, Yiwen, et al.
Pubblicazione: (2025)

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
di: Zhao, Yushu, et al.
Pubblicazione: (2025)

Active Learning Methods for Efficient Data Utilization and Model Performance Enhancement
di: Tseng, Chiung-Yi, et al.
Pubblicazione: (2025)

Mixture of Heterogeneous Grouped Experts for Language Modeling
di: Ma, Zhicheng, et al.
Pubblicazione: (2026)

L-MoE: End-to-End Training of a Lightweight Mixture of Low-Rank Adaptation Experts
di: Ji, Shihao, et al.
Pubblicazione: (2025)

MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts
di: Zhou, Jianan, et al.
Pubblicazione: (2024)

Mixtures of SubExperts for Large Language Continual Learning
di: Kang, Haeyong
Pubblicazione: (2025)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
di: He, Yifei, et al.
Pubblicazione: (2025)

MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts
di: Xie, Zhitian, et al.
Pubblicazione: (2024)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
di: Jin, Peng, et al.
Pubblicazione: (2024)

LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing
di: Hao, Jiawei, et al.
Pubblicazione: (2026)

Multimodal Representation Learning and Fusion
di: Jin, Qihang, et al.
Pubblicazione: (2025)

HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
di: Wei, Jia, et al.
Pubblicazione: (2026)

LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models
di: Hao, Qianyue, et al.
Pubblicazione: (2025)

HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts
di: Zhao, Hao, et al.
Pubblicazione: (2024)

Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models
di: Liang, Jingcong, et al.
Pubblicazione: (2025)

LLMSurgeon: Diagnosing Data Mixture of Large Language Models
di: Luo, Yaxin, et al.
Pubblicazione: (2026)

Modeling Spatio-temporal Dynamical Systems with Neural Discrete Learning and Levels-of-Experts
di: Wang, Kun, et al.
Pubblicazione: (2024)