Cover Image

Tensorized Clustered LoRA Merging for Multi-Task Interference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Su, Zhan, Mo, Fengran, Liang, Guojun, Zhang, Jinghan, Wen, Bingbing, Tiwari, Prayag, Nie, Jian-Yun
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mixture of Latent Experts Using Tensor Products
by: Su, Zhan, et al.
Published: (2024)

Parametric Retrieval-Augmented Generation using Latent Routing of LoRA Adapters
by: Su, Zhan, et al.
Published: (2025)

Towards Dynamic Dense Retrieval with Routing Strategy
by: Su, Zhan, et al.
Published: (2026)

Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Merging
by: Zhang, Haobo, et al.
Published: (2025)

LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks
by: Prabhakar, Akshara, et al.
Published: (2024)

LoRA on the Go: Instance-level Dynamic LoRA Selection and Merging
by: Lee, Seungeon, et al.
Published: (2025)

SCALE-LoRA: Auditing Post-Retrieval LoRA Composition with Residual Merging and View Reliability
by: Zhou, Shuaipeng, et al.
Published: (2026)

R-LoRA: Randomized Multi-Head LoRA for Efficient Multi-Task Learning
by: Liu, Jinda, et al.
Published: (2025)

Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering
by: Zhao, Ziyu, et al.
Published: (2024)

Block-Diagonal LoRA for Eliminating Communication Overhead in Tensor Parallel LoRA Serving
by: Wang, Xinyu, et al.
Published: (2025)

LoBAM: LoRA-Based Backdoor Attack on Model Merging
by: Yin, Ming, et al.
Published: (2024)

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning
by: Yang, Yaming, et al.
Published: (2024)

Label-Free Cross-Task LoRA Merging with Null-Space Compression
by: Lee, Wonyoung, et al.
Published: (2026)

CopRA: A Progressive LoRA Training Strategy
by: Zhuang, Zhan, et al.
Published: (2024)

MoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning
by: Ma, Yufei, et al.
Published: (2024)

IterIS: Iterative Inference-Solving Alignment for LoRA Merging
by: Chen, Hongxu, et al.
Published: (2024)

Run LoRA Run: Faster and Lighter LoRA Implementations
by: Cherniuk, Daria, et al.
Published: (2023)

Merge before Forget: A Single LoRA Continual Learning via Continual Merging
by: Qiao, Fuli, et al.
Published: (2025)

Adaptive LoRA Merge with Parameter Pruning for Low-Resource Generation
by: Miyano, Ryota, et al.
Published: (2025)

Higher-order Spatio-temporal Physics-incorporated Graph Neural Network for Multivariate Time Series Imputation
by: Liang, Guojun, et al.
Published: (2024)

DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Models
by: Zhang, Yuxuan, et al.
Published: (2024)

S-LoRA: Serving Thousands of Concurrent LoRA Adapters
by: Sheng, Ying, et al.
Published: (2023)

Decouple and Orthogonalize: A Data-Free Framework for LoRA Merging
by: Zheng, Shenghe, et al.
Published: (2025)

LoRA-MME: Multi-Model Ensemble of LoRA-Tuned Encoders for Code Comment Classification
by: Haider, Md Akib, et al.
Published: (2026)

ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs
by: Shah, Viraj, et al.
Published: (2023)

LoRA-FAIR: Federated LoRA Fine-Tuning with Aggregation and Initialization Refinement
by: Bian, Jieming, et al.
Published: (2024)

CE-LoRA: Computation-Efficient LoRA Fine-Tuning for Language Models
by: Chen, Guanduo, et al.
Published: (2025)

LoRA Diffusion: Zero-Shot LoRA Synthesis for Diffusion Model Personalization
by: Smith, Ethan, et al.
Published: (2024)

LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing
by: Li, Wenbing, et al.
Published: (2025)

FIM-LoRA: Task-Informative Rank Allocation for LoRA via Calibration-Time Gradient-Variance Estimation
by: Sathyavageeswaran, Ramakrishnan
Published: (2026)

Disentangling Task Conflicts in Multi-Task LoRA via Orthogonal Gradient Projection
by: Yang, Ziyu, et al.
Published: (2026)

LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluation
by: Zhou, Hongyun, et al.
Published: (2024)

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?
by: Wang, Zhengbo, et al.
Published: (2024)

Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning
by: Zhao, Ziyu, et al.
Published: (2025)

Flexi-LoRA with Input-Adaptive Ranks: Efficient Finetuning for Speech and Reasoning Tasks
by: Li, Zongqian, et al.
Published: (2026)

Effective LoRA Adapter Routing using Task Representations
by: Dhasade, Akash, et al.
Published: (2026)

FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
by: Zhang, Haoran, et al.
Published: (2026)

tLoRA: Efficient Multi-LoRA Training with Elastic Shared Super-Models
by: Li, Kevin, et al.
Published: (2026)

LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization
by: Yen, Jui-Nan, et al.
Published: (2024)

Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management
by: Zhang, Hang, et al.
Published: (2025)