Imagen de Portada

Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Chowdhury, Mohammed Nowaz Rabbani, Maghraoui, Kaoutar El, Tsai, Hsinyu, Wang, Naigang, Burr, Geoffrey W., Liu, Liu, Wang, Meng
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Robust Heterogeneous Analog-Digital Computing for Mixture-of-Experts Models with Theoretical Generalization Guarantees
por: Chowdhury, Mohammed Nowaz Rabbani, et al.
Publicado: (2026)

A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts
por: Chowdhury, Mohammed Nowaz Rabbani, et al.
Publicado: (2024)

Evaluating Fine-Tuned LLM Model For Medical Transcription With Small Low-Resource Languages Validated Dataset
por: Chowdhury, Mohammed Nowshad Ruhani, et al.
Publicado: (2026)

EvidenceMoE: A Physics-Guided Mixture-of-Experts with Evidential Critics for Advancing Fluorescence Light Detection and Ranging in Scattering Media
por: Erbas, Ismail, et al.
Publicado: (2025)

Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System
por: Fang, Yunhua, et al.
Publicado: (2025)

PHMForge: Evaluating LLM Agents on Industrial Prognostics through MCP-Native, Algorithm-Grounded Tools
por: Feng, Tianjun, et al.
Publicado: (2026)

On the Convergence Theory of Pipeline Gradient-based Analog In-memory Training
por: Wu, Zhaoxian, et al.
Publicado: (2024)

Paged Attention Meets FlexAttention: Unlocking Long-Context Efficiency in Deployed Inference
por: Joshi, Thomas, et al.
Publicado: (2025)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
por: Chu, Kexin, et al.
Publicado: (2025)

Context-Aware Mixture-of-Experts Inference on CXL-Enabled GPU-NDP Systems
por: Fan, Zehao, et al.
Publicado: (2025)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
por: He, Yifei, et al.
Publicado: (2025)

MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization
por: Zhang, Aozhong, et al.
Publicado: (2024)

MoQE: Improve Quantization Model performance via Mixture of Quantization Experts
por: Zhang, Jinhao, et al.
Publicado: (2025)

FLEx: Personalized Federated Learning for Mixture-of-Experts LLMs via Expert Grafting
por: Liu, Fan, et al.
Publicado: (2025)

MixtureKit: A General Framework for Composing, Training, and Visualizing Mixture-of-Experts Models
por: Chamma, Ahmad, et al.
Publicado: (2025)

Mixture of Diverse Size Experts
por: Sun, Manxi, et al.
Publicado: (2024)

CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initialization
por: Deng, Yanxia, et al.
Publicado: (2025)

Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelines
por: Merchant, Alimurtaza Mustafa, et al.
Publicado: (2026)

MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization
por: Zhao, Zhixiong, et al.
Publicado: (2026)

ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
por: He, Xin, et al.
Publicado: (2024)

Uncertainty Makes It Stable: Curiosity-Driven Quantized Mixture-of-Experts
por: Ordóñez, Sebastián Andrés Cajas, et al.
Publicado: (2025)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
por: Gao, Yuting, et al.
Publicado: (2025)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
por: Liu, Baihui, et al.
Publicado: (2026)

CompeteSMoE -- Statistically Guaranteed Mixture of Experts Training via Competition
por: Nguyen, Nam V., et al.
Publicado: (2025)

RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression
por: Zhong, Zhengjia, et al.
Publicado: (2026)

RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blocks
por: Liu, Ningyuan, et al.
Publicado: (2025)

Free Agent in Agent-Based Mixture-of-Experts Generative AI Framework
por: Liu, Jung-Hua
Publicado: (2025)

Mixture of Heterogeneous Grouped Experts for Language Modeling
por: Ma, Zhicheng, et al.
Publicado: (2026)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
por: Huang, Wei, et al.
Publicado: (2025)

TradExpert: Revolutionizing Trading with Mixture of Expert LLMs
por: Ding, Qianggang, et al.
Publicado: (2024)

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
por: Dwivedi, Chaitanya, et al.
Publicado: (2026)

Theoretical Analysis of Meta Reinforcement Learning: Generalization Bounds and Convergence Guarantees
por: Wang, Cangqing, et al.
Publicado: (2024)

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
por: Zhan, Zheng, et al.
Publicado: (2025)

Mixture-of-Experts Meets In-Context Reinforcement Learning
por: Wu, Wenhao, et al.
Publicado: (2025)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
por: Lu, Xudong, et al.
Publicado: (2024)

DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts
por: Feng, Jiarui, et al.
Publicado: (2026)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
por: Liu, Yilun, et al.
Publicado: (2025)

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
por: Liu, Yilun, et al.
Publicado: (2024)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
por: Yan, Jiaming, et al.
Publicado: (2025)

Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment
por: Liu, Zhili, et al.
Publicado: (2024)