Cover Image

HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wei, Jia, Zhang, Zhonghao, Chen, Ping, li, Qianyang, Pan, Yancheng, Wang, Shaoxun, Qiu, Ziyi, Wang, Longxiang
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

AWEMixer: Adaptive Wavelet-Enhanced Mixer Network for Long-Term Time Series Forecasting
by: Li, Qianyang, et al.
Published: (2025)

RepLoRA: Reparameterizing Low-Rank Adaptation via the Perspective of Mixture of Experts
by: Truong, Tuan, et al.
Published: (2025)

DPWMixer: Dual-Path Wavelet Mixer for Long-Term Time Series Forecasting
by: Qianyang, Li, et al.
Published: (2025)

DPANet: Dual Pyramid Attention Network for Multivariate Time Series Forecasting
by: Li, Qianyang, et al.
Published: (2025)

Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning
by: Zhao, Ziyu, et al.
Published: (2025)

Mixture of LoRA Experts
by: Wu, Xun, et al.
Published: (2024)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

ASGMamba: Adaptive Spectral Gating Mamba for Multivariate Time Series Forecasting
by: Li, Qianyang, et al.
Published: (2026)

TeamLoRA: Boosting Low-Rank Adaptation with Expert Collaboration and Competition
by: Lin, Tianwei, et al.
Published: (2024)

FLoE: Fisher-Based Layer Selection for Efficient Sparse Adaptation of Low-Rank Experts
by: Wang, Xinyi, et al.
Published: (2025)

Lifelong Knowledge Editing for Vision Language Models with Low-Rank Mixture-of-Experts
by: Chen, Qizhou, et al.
Published: (2024)

Adaptive Dilation Attention and Disentangled Mixture‐of‐Experts for Hot Metal Silicon Prediction in Blast Furnace Ironmaking Process
by: Danyi Wang, et al.
Published: (2026)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
by: Yang, Cheng, et al.
Published: (2024)

MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning
by: Wang, Xujia, et al.
Published: (2024)

A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
by: Sun, Mengyang, et al.
Published: (2025)

SAMoRA: Semantic-Aware Mixture of LoRA Experts for Task-Adaptive Learning
by: Shi, Boyan, et al.
Published: (2026)

The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level
by: Herbst, Jeremy, et al.
Published: (2026)

TalkLoRA: Communication-Aware Mixture of Low-Rank Adaptation for Large Language Models
by: Mu, Lin, et al.
Published: (2026)

SDGF: Fusing Static and Multi-Scale Dynamic Correlations for Multivariate Time Series Forecasting
by: Wang, Shaoxun, et al.
Published: (2025)

AdaMoLE: Fine-Tuning Large Language Models with Adaptive Mixture of Low-Rank Adaptation Experts
by: Liu, Zefang, et al.
Published: (2024)

Low-Rank Mixture-of-Experts for Continual Medical Image Segmentation
by: Chen, Qian, et al.
Published: (2024)

ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Experts
by: Du, Sinan, et al.
Published: (2024)

Mixture of Experts for Node Classification
by: Shi, Yu, et al.
Published: (2024)

L-MoE: End-to-End Training of a Lightweight Mixture of Low-Rank Adaptation Experts
by: Ji, Shihao, et al.
Published: (2025)

MINGLE: Mixture of Null-Space Gated Low-Rank Experts for Test-Time Continual Model Merging
by: Qiu, Zihuan, et al.
Published: (2025)

Hierarchical-Task-Aware Multi-modal Mixture of Incremental LoRA Experts for Embodied Continual Learning
by: Jia, Ziqi, et al.
Published: (2025)

MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning
by: Zhang, Jingfan, et al.
Published: (2024)

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
by: Wang, Zihan, et al.
Published: (2025)

Adaptive Shared Experts with LoRA-Based Mixture of Experts for Multi-Task Learning
by: Yang, Minghao, et al.
Published: (2025)

Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference
by: Yao, Jinghan, et al.
Published: (2024)

Multimodal Mixture of Low-Rank Experts for Sentiment Analysis and Emotion Recognition
by: Zhang, Shuo, et al.
Published: (2025)

Mixture of Low-Rank Adapter Experts in Generalizable Audio Deepfake Detection
by: Laakkonen, Janne, et al.
Published: (2025)

Multi-Task Dense Prediction via Mixture of Low-Rank Experts
by: Yang, Yuqi, et al.
Published: (2024)

Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation
by: Liu, Zhenyu, et al.
Published: (2025)

Adaptive Mixture of Low-Rank Experts for Robust Audio Spoofing Detection
by: Chen, Qixian, et al.
Published: (2025)

When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
by: Yoon, Youngsik, et al.
Published: (2026)

ASLoRA: Adaptive Sharing Low-Rank Adaptation Across Layers
by: Hu, Junyan, et al.
Published: (2024)

Mixture-of-Subspaces in Low-Rank Adaptation
by: Wu, Taiqiang, et al.
Published: (2024)

Mixture of LoRA Experts for Low-Resourced Multi-Accent Automatic Speech Recognition
by: Bagat, Raphaël, et al.
Published: (2025)

Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning
by: Zhao, Ziyu, et al.
Published: (2024)