Cover Image

Towards Specialized Generalists: A Multi-Task MoE-LoRA Framework for Domain-Specific LLM Adaptation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Yuxin, Zeng, Aoxiong, Yang, Xiangquan
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Disentangling Task Conflicts in Multi-Task LoRA via Orthogonal Gradient Projection
by: Yang, Ziyu, et al.
Published: (2026)

MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learning
by: Yang, Shu, et al.
Published: (2024)

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning
by: Yang, Yaming, et al.
Published: (2024)

MoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning
by: Ma, Yufei, et al.
Published: (2024)

Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
by: Li, Junzhuo, et al.
Published: (2025)

Hierarchical LoRA MoE for Efficient CTR Model Scaling
by: Zeng, Zhichen, et al.
Published: (2025)

DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Models
by: Zhang, Yuxuan, et al.
Published: (2024)

Brainstacks: Cross-Domain Cognitive Capabilities via Frozen MoE-LoRA Stacks for Continual LLM Learning
by: Ayyash, Mohammad R. Abu
Published: (2026)

LoRALib: A Standardized Benchmark for Evaluating LoRA-MoE Methods
by: Wang, Shaoheng, et al.
Published: (2025)

LoRA-Gen: Specializing Large Language Model via Online LoRA Generation
by: Xiao, Yicheng, et al.
Published: (2025)

AT-MoE: Adaptive Task-planning Mixture of Experts via LoRA Approach
by: Li, Xurui, et al.
Published: (2024)

LoRA on the Go: Instance-level Dynamic LoRA Selection and Merging
by: Lee, Seungeon, et al.
Published: (2025)

LoRA-Mini : Adaptation Matrices Decomposition and Selective Training
by: Singh, Ayush, et al.
Published: (2024)

LoRA+: Efficient Low Rank Adaptation of Large Models
by: Hayou, Soufiane, et al.
Published: (2024)

Noise-Robustness Through Noise: A Framework combining Asymmetric LoRA with Poisoning MoE
by: Wang, Zhaokun, et al.
Published: (2025)

The Long-Term Effects of Data Selection in LLM Fine-Tuning
by: Yang, Yuxin, et al.
Published: (2026)

LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization
by: Yen, Jui-Nan, et al.
Published: (2024)

MoSLD: An Extremely Parameter-Efficient Mixture-of-Shared LoRAs for Multi-Task Learning
by: Zhao, Lulu, et al.
Published: (2024)

A Note on LoRA
by: Fomenko, Vlad, et al.
Published: (2024)

LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades
by: Li, Yanan, et al.
Published: (2025)

LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild
by: Zhao, Ziyu, et al.
Published: (2024)

R-LoRA: Randomized Multi-Head LoRA for Efficient Multi-Task Learning
by: Liu, Jinda, et al.
Published: (2025)

KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillation
by: Azimi, Rambod, et al.
Published: (2024)

LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks
by: Fleshman, William, et al.
Published: (2025)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
by: Manzoni, Andrea
Published: (2026)

LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
by: Bałazy, Klaudia, et al.
Published: (2024)

Improving LoRA with Variational Learning
by: Cong, Bai, et al.
Published: (2025)

AirLLM: Diffusion Policy-based Adaptive LoRA for Remote Fine-Tuning of LLM over the Air
by: Yang, Shiyi, et al.
Published: (2025)

A Sensitivity-Driven Expert Allocation Method in LoRA-MoE for Efficient Fine-Tuning
by: Xu, Junzhou, et al.
Published: (2025)

TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
by: Kunwar, Pradip, et al.
Published: (2025)

The Impact of Initialization on LoRA Finetuning Dynamics
by: Hayou, Soufiane, et al.
Published: (2024)

LoRA Learns Less and Forgets Less
by: Biderman, Dan, et al.
Published: (2024)

LoRA Is Slower Than You Think
by: Ko, Seokmin
Published: (2025)

Task-Aware LoRA Adapter Composition via Similarity Retrieval in Vector Databases
by: Adsul, Riya, et al.
Published: (2026)

LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models
by: Elesedy, Hayder, et al.
Published: (2024)

Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA
by: Yang, Nuocheng, et al.
Published: (2026)

A Survey on LoRA of Large Language Models
by: Mao, Yuren, et al.
Published: (2024)

Tina: Tiny Reasoning Models via LoRA
by: Wang, Shangshang, et al.
Published: (2025)

Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning
by: Lee, Yu-Ang, et al.
Published: (2026)

RadLite: Multi-Task LoRA Fine-Tuning of Small Language Models for CPU-Deployable Radiology AI
by: Gupta, Pankaj, et al.
Published: (2026)