Image de couverture de livre

EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Fu, Zhongqian, Zhao, Tianyi, Ding, Ning, Yu, Xianzhi, Li, Xiaosong, Tang, Yehui, Wang, Yunhe
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
par: Tang, Yehui, et autres
Publié: (2025)

Post-Trained MoE Can Skip Half Experts via Self-Distillation
par: Lv, Xingtai, et autres
Publié: (2026)

Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs
par: Tang, Yehui, et autres
Publié: (2025)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
par: Wu, Haoyuan, et autres
Publié: (2025)

QuantMoE-Bench: Examining Post-Training Quantization for Mixture-of-Experts
par: Li, Pingzhi, et autres
Publié: (2024)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
par: Xia, Xinfeng, et autres
Publié: (2025)

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
par: Deng, Jianing, et autres
Publié: (2026)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
par: Qian, Yulei, et autres
Publié: (2024)

GPT4Image: Large Pre-trained Models Help Vision Models Learn Better on Perception Task
par: Ding, Ning, et autres
Publié: (2023)

LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
par: Qu, Xiaoye, et autres
Publié: (2024)

DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts
par: Feng, Yuchen, et autres
Publié: (2025)

Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
par: Li, Pingzhi, et autres
Publié: (2025)

Steering MoE LLMs via Expert (De)Activation
par: Fayyaz, Mohsen, et autres
Publié: (2025)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
par: Li, Zichong, et autres
Publié: (2025)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
par: Takashiro, Shota, et autres
Publié: (2026)

MH-MoE: Multi-Head Mixture-of-Experts
par: Huang, Shaohan, et autres
Publié: (2024)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
par: Liu, Baihui, et autres
Publié: (2026)

Mixture of Lookup Experts
par: Jie, Shibo, et autres
Publié: (2025)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
par: Gu, Naibin, et autres
Publié: (2025)

Training Report of TeleChat3-MoE
par: Liu, Xinzhang, et autres
Publié: (2025)

S2MoE: Robust Sparse Mixture of Experts via Stochastic Learning
par: Do, Giang, et autres
Publié: (2025)

Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
par: Wei, Tianwen, et autres
Publié: (2024)

Advancing Expert Specialization for Better MoE
par: Guo, Hongcan, et autres
Publié: (2025)

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
par: Shi, Jingze, et autres
Publié: (2026)

Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats
par: Zhang, Manyi, et autres
Publié: (2026)

Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping
par: Ding, Ning, et autres
Publié: (2025)

Enhancing Post-Training Quantization via Future Activation Awareness
par: Lv, Zheqi, et autres
Publié: (2026)

GEM: GPU-Variability-Aware Expert to GPU Mapping for MoE Systems
par: Wawdhane, Sourish, et autres
Publié: (2026)

$\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts
par: Chen, Guanjie, et autres
Publié: (2024)

An Empirical Study of World Model Quantization
par: Fu, Zhongqian, et autres
Publié: (2026)

GuiLoMo: Allocating Expert Number and Rank for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectors
par: Zhang, Hengyuan, et autres
Publié: (2025)

Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse
par: Fu, Zizhuo, et autres
Publié: (2026)

Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
par: Jiang, Fan, et autres
Publié: (2026)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
par: Muzio, Alexandre, et autres
Publié: (2024)

ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns
par: Zhao, Ziyu, et autres
Publié: (2026)

Do Domain-specific Experts exist in MoE-based LLMs?
par: Do, Giang, et autres
Publié: (2026)

Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning
par: Bi, Zhenni, et autres
Publié: (2024)

TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Acceleration
par: Wei, Linye, et autres
Publié: (2026)

MoE-DiffuSeq: Enhancing Long-Document Diffusion Models with Sparse Attention and Mixture of Experts
par: Christoforos, Alexandros, et autres
Publié: (2025)

LLaDA-MoE: A Sparse MoE Diffusion Language Model
par: Zhu, Fengqi, et autres
Publié: (2025)