Image de couverture de livre

Faster MoE LLM Inference for Extremely Large Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yang, Haoqi, Shi, Luohe, Li, Qiwei, Li, Zuchao, Wang, Ping, Du, Bo, Shen, Mengjia, Zhao, Hai
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models
par: Shi, Luohe, et autres
Publié: (2024)

Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption
par: Shi, Luohe, et autres
Publié: (2024)

Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios
par: Shi, Luohe, et autres
Publié: (2025)

From Parameters to Performance: A Data-Driven Study on LLM Structure and Development
par: Wang, Suqing, et autres
Publié: (2025)

IAM: Efficient Inference through Attention Mapping between Different-scale LLMs
par: Zhao, Yi, et autres
Publié: (2025)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
par: Song, Weixi, et autres
Publié: (2023)

KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embedding
par: Shi, Luohe, et autres
Publié: (2025)

LocMoE: A Low-Overhead MoE for Large Language Model Training
par: Li, Jing, et autres
Publié: (2024)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
par: Li, Zichong, et autres
Publié: (2025)

MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
par: Li, Shuhuai, et autres
Publié: (2026)

Symbiotic-MoE: Unlocking the Synergy between Generation and Understanding
par: Liu, Xiangyue, et autres
Publié: (2026)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
par: Xia, Xinfeng, et autres
Publié: (2025)

SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
par: Tang, Shengkun, et autres
Publié: (2026)

SirLLM: Streaming Infinite Retentive LLM
par: Yao, Yao, et autres
Publié: (2024)

Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
par: Ma, Wenhan, et autres
Publié: (2025)

Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast
par: Shi, Chufan, et autres
Publié: (2024)

SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layers
par: Tang, Zicong, et autres
Publié: (2025)

Deconstructing Pre-training: Knowledge Attribution Analysis in MoE and Dense Models
par: Wang, Bo, et autres
Publié: (2026)

Model Hemorrhage and the Robustness Limits of Large Language Models
par: Ma, Ziyang, et autres
Publié: (2025)

Yuan3.0 Ultra: A Trillion-Parameter Enterprise-Oriented MoE LLM
par: ai, YuanLab., et autres
Publié: (2026)

Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
par: Li, Bo, et autres
Publié: (2026)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
par: Manzoni, Andrea
Publié: (2026)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
par: Qian, Yulei, et autres
Publié: (2024)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
par: Gu, Naibin, et autres
Publié: (2025)

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
par: Cao, Mingyu, et autres
Publié: (2024)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
par: Liu, Baihui, et autres
Publié: (2026)

Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts
par: Kang, Junmo, et autres
Publié: (2024)

XQuant: Achieving Ultra-Low Bit KV Cache Quantization with Cross-Layer Compression
par: Yang, Haoqi, et autres
Publié: (2025)

GRIN: GRadient-INformed MoE
par: Liu, Liyuan, et autres
Publié: (2024)

GKT: A Novel Guidance-Based Knowledge Transfer Framework For Efficient Cloud-edge Collaboration LLM Deployment
par: Yao, Yao, et autres
Publié: (2024)

FlashDecoding++: Faster Large Language Model Inference on GPUs
par: Hong, Ke, et autres
Publié: (2023)

Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
par: Li, Junzhuo, et autres
Publié: (2025)

Segment First or Comprehend First? Explore the Limit of Unsupervised Word Segmentation with Large Language Models
par: Zhang, Zihong, et autres
Publié: (2025)

What Gets Activated: Uncovering Domain and Driver Experts in MoE Language Models
par: Hu, Guimin, et autres
Publié: (2026)

Cascade Speculative Drafting for Even Faster LLM Inference
par: Chen, Ziyi, et autres
Publié: (2023)

Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoE
par: Huang, Haiduo, et autres
Publié: (2025)

Harder Tasks Need More Experts: Dynamic Routing in MoE Models
par: Huang, Quzhe, et autres
Publié: (2024)

$\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts
par: Chen, Guanjie, et autres
Publié: (2024)

Mix- and MoE-DPO: A Variational Inference Approach to Direct Preference Optimization
par: Bohne, Jason, et autres
Publié: (2025)

Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models
par: Yao, Yao, et autres
Publié: (2023)