Cover Image

Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Feng, Wenfeng, Hao, Chuzhan, Zhang, Yuewei, Han, Yu, Wang, Hao
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RASD: Retrieval-Augmented Speculative Decoding
by: Quan, Guofeng, et al.
Published: (2025)

Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
by: Ban, Hao, et al.
Published: (2025)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

DynaSearcher: Dynamic Knowledge Graph Augmented Search Agent via Multi-Reward Reinforcement Learning
by: Hao, Chuzhan, et al.
Published: (2025)

MoSLD: An Extremely Parameter-Efficient Mixture-of-Shared LoRAs for Multi-Task Learning
by: Zhao, Lulu, et al.
Published: (2024)

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
by: Li, Dengchun, et al.
Published: (2024)

Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
by: Microsoft, et al.
Published: (2025)

VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Models
by: Jiang, Guochao, et al.
Published: (2025)

Position: Pause Recycling LoRAs and Prioritize Mechanisms to Uncover Limits and Effectiveness
by: Chen, Mei-Yen, et al.
Published: (2025)

Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering
by: Zhao, Ziyu, et al.
Published: (2024)

STAR: Constraint LoRA with Dynamic Active Learning for Data-Efficient Fine-Tuning of Large Language Models
by: Zhang, Linhai, et al.
Published: (2024)

ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion
by: Khan, Rana Muhammad Shahroz, et al.
Published: (2025)

LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades
by: Li, Yanan, et al.
Published: (2025)

AirRAG: Autonomous Strategic Planning and Reasoning Steer Retrieval Augmented Generation
by: Feng, Wenfeng, et al.
Published: (2025)

LoRA-Gen: Specializing Large Language Model via Online LoRA Generation
by: Xiao, Yicheng, et al.
Published: (2025)

LoRA-Squeeze: Simple and Effective Post-Tuning and In-Tuning Compression of LoRA Modules
by: Vulić, Ivan, et al.
Published: (2026)

DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Models
by: Zhang, Yuxuan, et al.
Published: (2024)

HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning
by: Tian, Chunlin, et al.
Published: (2024)

ELDER: Enhancing Lifelong Model Editing with Mixture-of-LoRA
by: Li, Jiaang, et al.
Published: (2024)

FlowKV: A Disaggregated Inference Framework with Low-Latency KV Cache Transfer and Load-Aware Scheduling
by: Li, Weiqing, et al.
Published: (2025)

Learning Attentional Mixture of LoRAs for Language Model Continual Learning
by: Liu, Jialin, et al.
Published: (2024)

DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanism
by: Li, Dengchun, et al.
Published: (2025)

LoRA+: Efficient Low Rank Adaptation of Large Models
by: Hayou, Soufiane, et al.
Published: (2024)

A Survey on LoRA of Large Language Models
by: Mao, Yuren, et al.
Published: (2024)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
by: Zhang, Jun, et al.
Published: (2025)

Subject or Style: Adaptive and Training-Free Mixture of LoRAs
by: Zhang, Jia-Chen, et al.
Published: (2025)

KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillation
by: Azimi, Rambod, et al.
Published: (2024)

HypeLoRA: Hyper-Network-Generated LoRA Adapters for Calibrated Language Model Fine-Tuning
by: Trojan, Bartosz, et al.
Published: (2026)

Beyond Stochastic Exploration: What Makes Training Data Valuable for Agentic Search
by: Hao, Chuzhan, et al.
Published: (2026)

PVPO: Pre-Estimated Value-Based Policy Optimization for Agentic Reasoning
by: Feng, Wenfeng, et al.
Published: (2025)

LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models
by: Ran, Delong, et al.
Published: (2025)

MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Models
by: Cao, Jie, et al.
Published: (2025)

LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models
by: Elesedy, Hayder, et al.
Published: (2024)

Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing
by: Cheng, Ming, et al.
Published: (2025)

In-Context Meta LoRA Generation
by: Shao, Yihua, et al.
Published: (2025)

SAMoRA: Semantic-Aware Mixture of LoRA Experts for Task-Adaptive Learning
by: Shi, Boyan, et al.
Published: (2026)

A Note on LoRA
by: Fomenko, Vlad, et al.
Published: (2024)

HSKBenchmark: Modeling and Benchmarking Chinese Second Language Acquisition in Large Language Models through Curriculum Tuning
by: Yang, Qihao, et al.
Published: (2025)

LD-MoLE: Learnable Dynamic Routing for Mixture of LoRA Experts
by: Zhuang, Yuan, et al.
Published: (2025)

A Language-Guided Bayesian Optimization for Efficient LoRA Hyperparameter Search
by: Seong-Eun, Baek, et al.
Published: (2026)