Imagen de Portada

Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Liu, Zhenyu, Liu, Yunzhen, Fan, Zehao, Gagnon, Garrett, Hou, Yayue, Wu, Nan, Kang, Yangwook, Liu, Liu
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Context-Aware Mixture-of-Experts Inference on CXL-Enabled GPU-NDP Systems
por: Fan, Zehao, et al.
Publicado: (2025)

Sparse Attention Remapping with Clustering for Efficient LLM Decoding on PIM
por: Fan, Zehao, et al.
Publicado: (2025)

Enhance DNN Adversarial Robustness and Efficiency via Injecting Noise to Non-Essential Neurons
por: Liu, Zhenyu, et al.
Publicado: (2024)

TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling
por: Gu, Hongyaoxing, et al.
Publicado: (2026)

MoRE: A Mixture of Low-Rank Experts for Adaptive Multi-Task Learning
por: Zhang, Dacao, et al.
Publicado: (2025)

MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensators
por: Huang, Beichen, et al.
Publicado: (2025)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
por: Tang, Chuanyu, et al.
Publicado: (2024)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
por: Yan, Jiaming, et al.
Publicado: (2025)

Spatial-Temporal Mixture-of-Graph-Experts for Multi-Type Crime Prediction
por: Wu, Ziyang, et al.
Publicado: (2024)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
por: He, Yifei, et al.
Publicado: (2025)

RepLoRA: Reparameterizing Low-Rank Adaptation via the Perspective of Mixture of Experts
por: Truong, Tuan, et al.
Publicado: (2025)

RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
por: Sun, Mengyang, et al.
Publicado: (2026)

Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts
por: Li, Cheng, et al.
Publicado: (2025)

LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training
por: Liu, Xinyi, et al.
Publicado: (2026)

CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving
por: Zhao, Adrian, et al.
Publicado: (2026)

Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees
por: Chowdhury, Mohammed Nowaz Rabbani, et al.
Publicado: (2026)

Efficient Handwriting-Based Alzheimer,s Disease Diagnosis Using a Low-Rank Mixture of Experts Deep Learning Framework
por: Wang, Wu, et al.
Publicado: (2026)

Efficient Low-Tubal-Rank Tensor Estimation via Alternating Preconditioned Gradient Descent
por: Liu, Zhiyu, et al.
Publicado: (2025)

Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning
por: Cong, Peizhuang, et al.
Publicado: (2025)

Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients
por: Zhang, Zhenyu, et al.
Publicado: (2024)

Mixture of Low Rank Adaptation with Partial Parameter Sharing for Time Series Forecasting
por: Pan, Licheng, et al.
Publicado: (2025)

Gaussian Process-Gated Hierarchical Mixtures of Experts
por: Liu, Yuhao, et al.
Publicado: (2023)

HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
por: Wei, Jia, et al.
Publicado: (2026)

MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Models
por: Liu, Zehua, et al.
Publicado: (2025)

Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion
por: Tang, Anke, et al.
Publicado: (2024)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
por: Huang, Wei, et al.
Publicado: (2025)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
por: Huang, Wei, et al.
Publicado: (2024)

Maximum Score Routing For Mixture-of-Experts
por: Dong, Bowen, et al.
Publicado: (2025)

A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
por: Sun, Mengyang, et al.
Publicado: (2025)

L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts
por: Yang, Minghao, et al.
Publicado: (2026)

Mixture of Length and Pruning Experts for Knowledge Graphs Reasoning
por: Du, Enjun, et al.
Publicado: (2025)

RoME: Domain-Robust Mixture-of-Experts for MILP Solution Prediction across Domains
por: Pu, Tianle, et al.
Publicado: (2025)

MixTTE: Multi-Level Mixture-of-Experts for Scalable and Adaptive Travel Time Estimation
por: Jiang, Wenzhao, et al.
Publicado: (2026)

Mixture-of-Subspaces in Low-Rank Adaptation
por: Wu, Taiqiang, et al.
Publicado: (2024)

Variational Inference, Entropy, and Orthogonality: A Unified Theory of Mixture-of-Experts
por: Su, Ye, et al.
Publicado: (2026)

FreqMoE: Enhancing Time Series Forecasting through Frequency Decomposition Mixture of Experts
por: Liu, Ziqi
Publicado: (2025)

Mixture of Low-Rank Adapter Experts in Generalizable Audio Deepfake Detection
por: Laakkonen, Janne, et al.
Publicado: (2025)

Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization
por: Chu, Yunlong, et al.
Publicado: (2025)

Efficient Generalized Low-Rank Tensor Contextual Bandits
por: Yi, Qianxin, et al.
Publicado: (2023)

Mixture of Universal Experts: Scaling Virtual Width via Depth-Width Transformation
por: Chen, Yilong, et al.
Publicado: (2026)