Cover Image

MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Luo, Tongxu, Lei, Jiahe, Lei, Fangyu, Liu, Weihao, He, Shizhu, Zhao, Jun, Liu, Kang
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Models
by: Lei, Fangyu, et al.
Published: (2023)

TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
by: Kunwar, Pradip, et al.
Published: (2025)

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
by: Li, Dengchun, et al.
Published: (2024)

AdaMoLE: Fine-Tuning Large Language Models with Adaptive Mixture of Low-Rank Adaptation Experts
by: Liu, Zefang, et al.
Published: (2024)

Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent
by: Yu, Xiaoyan, et al.
Published: (2024)

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
by: Liu, Yilun, et al.
Published: (2024)

DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanism
by: Li, Dengchun, et al.
Published: (2025)

CoMoE: Contrastive Representation for Mixture-of-Experts in Parameter-Efficient Fine-tuning
by: Feng, Jinyuan, et al.
Published: (2025)

MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Models
by: Cao, Jie, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
by: Zhang, Buze, et al.
Published: (2026)

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
by: Jiang, Ting, et al.
Published: (2024)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
by: Liu, Yilun, et al.
Published: (2025)

Adaptive and Fine-grained Module-wise Expert Pruning for Efficient LoRA-MoE Fine-Tuning
by: Li, Weihang, et al.
Published: (2026)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

TLoRA+: A Low-Rank Parameter-Efficient Fine-Tuning Method for Large Language Models
by: Cao, Yarui, et al.
Published: (2026)

MoDE: Effective Multi-task Parameter Efficient Fine-Tuning with a Mixture of Dyadic Experts
by: Ning, Lin, et al.
Published: (2024)

MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Models
by: Liu, Zehua, et al.
Published: (2025)

DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models
by: Huang, Yiming, et al.
Published: (2024)

MLAE: Masked LoRA Experts for Visual Parameter-Efficient Fine-Tuning
by: Wang, Junjie, et al.
Published: (2024)

SPMTrack: Spatio-Temporal Parameter-Efficient Fine-Tuning with Mixture of Experts for Scalable Visual Tracking
by: Cai, Wenrui, et al.
Published: (2025)

HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
by: Lin, Zheng, et al.
Published: (2025)

S$^3$HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering
by: Lei, Fangyu, et al.
Published: (2023)

Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning
by: Wang, Lei, et al.
Published: (2025)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
by: Manzoni, Andrea
Published: (2026)

S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning
by: Zeng, Hanqing, et al.
Published: (2025)

DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model
by: Gao, Chao, et al.
Published: (2024)

RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blocks
by: Liu, Ningyuan, et al.
Published: (2025)

Neural Incompatibility: The Unbridgeable Gap of Cross-Scale Parametric Knowledge Transfer in Large Language Models
by: Tan, Yuqiao, et al.
Published: (2025)

Unveiling Super Experts in Mixture-of-Experts Large Language Models
by: Su, Zunhai, et al.
Published: (2025)

HDMoLE: Mixture of LoRA Experts with Hierarchical Routing and Dynamic Thresholds for Fine-Tuning LLM-based ASR Models
by: Mu, Bingshen, et al.
Published: (2024)

Shuttle Between the Instructions and the Parameters of Large Language Models
by: Sun, Wangtao, et al.
Published: (2025)

MoPE: Mixture of Prompt Experts for Parameter-Efficient and Scalable Multimodal Fusion
by: Jiang, Ruixiang, et al.
Published: (2024)

Generalizable Foundation Models for Calorimetry via Mixtures-of-Experts and Parameter Efficient Fine Tuning
by: Cardona-Giraldo, Carlos, et al.
Published: (2026)

A Sensitivity-Driven Expert Allocation Method in LoRA-MoE for Efficient Fine-Tuning
by: Xu, Junzhou, et al.
Published: (2025)

Reasoning-Table: Exploring Reinforcement Learning for Table Reasoning
by: Lei, Fangyu, et al.
Published: (2025)

EcoLoRA: Communication-Efficient Federated Fine-Tuning of Large Language Models
by: Liu, Han, et al.
Published: (2025)

Geo-Expert: Towards Expert-Level Geological Reasoning via Parameter-Efficient Fine-Tuning
by: Guo, Chenyou, et al.
Published: (2026)

SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
by: Lin, Zheng, et al.
Published: (2024)

AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models
by: Liu, Zeyu, et al.
Published: (2024)

GATE: Graph-based Adaptive Tool Evolution Across Diverse Tasks
by: Luo, Jianwen, et al.
Published: (2025)