Image de couverture de livre

MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Duanmu, Haojie, Li, Xiuhong, Yuan, Zhihang, Zheng, Size, Duan, Jiangfei, Zhang, Xingcheng, Lin, Dahua
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models
par: Duanmu, Haojie, et autres
Publié: (2024)

MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
par: Duan, Jiangfei, et autres
Publié: (2024)

DynaMo: Runtime Switchable Quantization for MoE with Cross-Dataset Adaptation
par: Zheng, Zihao, et autres
Publié: (2025)

Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
par: Li, Bo, et autres
Publié: (2026)

MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints
par: Yuan, Yichao, et autres
Publié: (2025)

KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models
par: Xu, Zukang, et autres
Publié: (2026)

LLaDA-MoE: A Sparse MoE Diffusion Language Model
par: Zhu, Fengqi, et autres
Publié: (2025)

VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models
par: Qin, Guangshuo, et autres
Publié: (2026)

MoE-PHDS: One MoE checkpoint for flexible runtime sparsity
par: Hannah, Lauren. A, et autres
Publié: (2025)

WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
par: Yue, Yuxuan, et autres
Publié: (2024)

BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization
par: Zhao, Jiayu, et autres
Publié: (2026)

NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decoding
par: Chen, Jiefei, et autres
Publié: (2026)

CoMoE: Collaborative Optimization of Expert Aggregation and Offloading for MoE-based LLMs at Edge
par: Li, Muqing, et autres
Publié: (2025)

GW-MoE: Resolving Uncertainty in MoE Router with Global Workspace Theory
par: Wu, Haoze, et autres
Publié: (2024)

Revisiting MoE and Dense Speed-Accuracy Comparisons for LLM Training
par: Du, Xianzhi, et autres
Publié: (2024)

Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
par: Zeng, Zhiyuan, et autres
Publié: (2024)

DTop-p MoE: Sparsity-Controlled Dynamic Top-p MoE for Foundation Model Pre-training
par: Jin, Can, et autres
Publié: (2025)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
par: Cao, Shiyi, et autres
Publié: (2024)

MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
par: Li, Shuhuai, et autres
Publié: (2026)

LocMoE: A Low-Overhead MoE for Large Language Model Training
par: Li, Jing, et autres
Publié: (2024)

SMoE: An Algorithm-System Co-Design for Pushing MoE to the Edge via Expert Substitution
par: Zhu, Guoying, et autres
Publié: (2025)

UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training
par: Zheng, Size, et autres
Publié: (2026)

GRIN: GRadient-INformed MoE
par: Liu, Liyuan, et autres
Publié: (2024)

DOT-MoE: Differentiable Optimal Transport for MoEfication
par: Bamba, Udbhav, et autres
Publié: (2026)

GazeFormer-MoE: Context-Aware Gaze Estimation via CLIP and MoE Transformer
par: Zhao, Xinyuan, et autres
Publié: (2026)

DyMoE: Dynamic Expert Orchestration with Mixed-Precision Quantization for Efficient MoE Inference on Edge
par: Huang, Yuegui, et autres
Publié: (2026)

Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs
par: Zhou, Xin, et autres
Publié: (2024)

Sigma-MoE-Tiny Technical Report
par: Hu, Qingguo, et autres
Publié: (2025)

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
par: Shi, Jingze, et autres
Publié: (2026)

SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations
par: Guo, Wentao, et autres
Publié: (2025)

6Bit-Diffusion: Inference-Time Mixed-Precision Quantization for Video Diffusion Models
par: Su, Rundong, et autres
Publié: (2026)

SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification
par: Lai, Zhenglin, et autres
Publié: (2025)

RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression
par: Zhong, Zhengjia, et autres
Publié: (2026)

Expert Divergence Learning for MoE-based Language Models
par: Li, Jiaang, et autres
Publié: (2026)

MoETTA: Test-Time Adaptation Under Mixed Distribution Shifts with MoE-LayerNorm
par: Fan, Xiao, et autres
Publié: (2025)

AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies
par: Zhang, Bo-Wen, et autres
Publié: (2024)

The Empirical Impact of Reducing Symmetries on the Performance of Deep Ensembles and MoE
par: Chernov, Andrei, et autres
Publié: (2025)

DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism
par: Zeng, Zhichen, et autres
Publié: (2026)

D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
par: Wang, Haodong, et autres
Publié: (2025)

MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE
par: Huang, Zongle, et autres
Publié: (2025)