Cover Image

Learning Attentional Mixture of LoRAs for Language Model Continual Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Jialin, Wu, Jianhua, Liu, Jie, Duan, Yutai
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Subject or Style: Adaptive and Training-Free Mixture of LoRAs
by: Zhang, Jia-Chen, et al.
Published: (2025)

Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models
by: Feng, Wenfeng, et al.
Published: (2024)

Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing
by: Cheng, Ming, et al.
Published: (2025)

MoSLD: An Extremely Parameter-Efficient Mixture-of-Shared LoRAs for Multi-Task Learning
by: Zhao, Lulu, et al.
Published: (2024)

LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks
by: Prabhakar, Akshara, et al.
Published: (2024)

Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
by: Microsoft, et al.
Published: (2025)

ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning
by: Qiu, Ruizhong, et al.
Published: (2026)

Towards Modular LLMs by Building and Reusing a Library of LoRAs
by: Ostapenko, Oleksiy, et al.
Published: (2024)

Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering
by: Zhao, Ziyu, et al.
Published: (2024)

qa-FLoRA: Data-free query-adaptive Fusion of LoRAs for LLMs
by: Shukla, Shreya, et al.
Published: (2025)

Position: Pause Recycling LoRAs and Prioritize Mechanisms to Uncover Limits and Effectiveness
by: Chen, Mei-Yen, et al.
Published: (2025)

Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
by: Ban, Hao, et al.
Published: (2025)

Mixture of LoRA Experts
by: Wu, Xun, et al.
Published: (2024)

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
by: Li, Dengchun, et al.
Published: (2024)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

Routing Distilled Knowledge via Mixture of LoRA Experts for Large Language Model based Bundle Generation
by: Feng, Kaidong, et al.
Published: (2025)

LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks
by: Wang, Hanqing, et al.
Published: (2024)

Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
by: Fan, Chenghao, et al.
Published: (2025)

ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion
by: Khan, Rana Muhammad Shahroz, et al.
Published: (2025)

SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture
by: Han, Jiayi, et al.
Published: (2024)

SAMoRA: Semantic-Aware Mixture of LoRA Experts for Task-Adaptive Learning
by: Shi, Boyan, et al.
Published: (2026)

Align, Don't Divide: Revisiting the LoRA Architecture in Multi-Task Learning
by: Liu, Jinda, et al.
Published: (2025)

Improving Recursive Transformers with Mixture of LoRAs
by: Nouriborji, Mohammadmahdi, et al.
Published: (2025)

Assertion Detection Large Language Model In-context Learning LoRA Fine-tuning
by: Ji, Yuelyu, et al.
Published: (2024)

Lamer-SSL: Layer-aware Mixture of LoRA Experts for Continual Multilingual Expansion of Self-supervised Models without Forgetting
by: Xu, Jing, et al.
Published: (2026)

HyperDet: Generalizable Detection of Synthesized Images by Generating and Merging A Mixture of Hyper LoRAs
by: Cao, Huangsen, et al.
Published: (2024)

Personalized Federated Fine-tuning for Heterogeneous Data: An Automatic Rank Learning Approach via Two-Level LoRA
by: Hao, Jie, et al.
Published: (2025)

ELDER: Enhancing Lifelong Model Editing with Mixture-of-LoRA
by: Li, Jiaang, et al.
Published: (2024)

Separating Shared and Domain-Specific LoRAs for Multi-Domain Learning
by: Takama, Yusaku, et al.
Published: (2025)

MeTA-LoRA: Data-Efficient Multi-Task Fine-Tuning for Large Language Models
by: Cheng, Bo, et al.
Published: (2025)

LoRA-Gen: Specializing Large Language Model via Online LoRA Generation
by: Xiao, Yicheng, et al.
Published: (2025)

Promoting Data and Model Privacy in Federated Learning through Quantized LoRA
by: Zhu, JianHao, et al.
Published: (2024)

Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning
by: Xia, Wenhan, et al.
Published: (2024)

STAR: Constraint LoRA with Dynamic Active Learning for Data-Efficient Fine-Tuning of Large Language Models
by: Zhang, Linhai, et al.
Published: (2024)

P-React: Synthesizing Topic-Adaptive Reactions of Personality Traits via Mixture of Specialized LoRA Experts
by: Dan, Yuhao, et al.
Published: (2024)

Improving LoRA with Variational Learning
by: Cong, Bai, et al.
Published: (2025)

CoMoL: Efficient Mixture of LoRA Experts via Dynamic Core Space Merging
by: Cao, Jie, et al.
Published: (2026)

Dual LoRA: Enhancing LoRA with Magnitude and Direction Updates
by: Xu, Yixing, et al.
Published: (2025)

Fusian: Multi-LoRA Fusion for Fine-Grained Continuous MBTI Personality Control in Large Language Models
by: Chen, Zehao, et al.
Published: (2026)

DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanism
by: Li, Dengchun, et al.
Published: (2025)