Cover Image

Put the Space of LoRA Initialization to the Extreme to Preserve Pre-trained Knowledge

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tang, Pengwei, Hu, Xiaolin, Liu, Yong, Ding, Lizhong, Zhang, Dongjie, Wu, Xing, Zhang, Debing
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

$D^2LoRA$: Data-Driven LoRA Initialization for Low Resource Tasks
by: SeraJ, Javad, et al.
Published: (2025)

The Impact of Initialization on LoRA Finetuning Dynamics
by: Hayou, Soufiane, et al.
Published: (2024)

QuAILoRA: Quantization-Aware Initialization for LoRA
by: Lawton, Neal, et al.
Published: (2024)

Crowded in B-Space: Calibrating Shared Directions for LoRA Merging
by: Tang, Yixuan, et al.
Published: (2026)

Dual LoRA: Enhancing LoRA with Magnitude and Direction Updates
by: Xu, Yixing, et al.
Published: (2025)

Vision as LoRA
by: Wang, Han, et al.
Published: (2025)

ADePT: Adaptive Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning
by: Tang, Pengwei, et al.
Published: (2025)

Enhancing Learned Knowledge in LoRA Adapters Through Efficient Contrastive Decoding on Ascend NPUs
by: Heisler, Morgan Lindsay, et al.
Published: (2025)

LoRA-Drop: Temporal LoRA Decoding for Efficient LLM Inference
by: Rajabzadeh, Hossein, et al.
Published: (2026)

KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillation
by: Azimi, Rambod, et al.
Published: (2024)

Mixture of LoRA Experts
by: Wu, Xun, et al.
Published: (2024)

LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks
by: Wang, Hanqing, et al.
Published: (2024)

CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices
by: Zhao, Weilin, et al.
Published: (2023)

Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering
by: Zhao, Ziyu, et al.
Published: (2024)

LoRA-PAR: A Flexible Dual-System LoRA Partitioning Approach to Efficient LLM Fine-Tuning
by: Huang, Yining, et al.
Published: (2025)

DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models
by: Deng, Guanzhi, et al.
Published: (2026)

VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks
by: Li, Yang, et al.
Published: (2024)

In-Context Meta LoRA Generation
by: Shao, Yihua, et al.
Published: (2025)

LoRA on the Go: Instance-level Dynamic LoRA Selection and Merging
by: Lee, Seungeon, et al.
Published: (2025)

LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluation
by: Zhou, Hongyun, et al.
Published: (2024)

Personalized LoRA for Human-Centered Text Understanding
by: Zhang, You, et al.
Published: (2024)

PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
by: Meng, Xiangdi, et al.
Published: (2024)

DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Models
by: Zhang, Yuxuan, et al.
Published: (2024)

Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
by: Fan, Chenghao, et al.
Published: (2025)

AFA-LoRA: Enabling Non-Linear Adaptations in LoRA with Activation Function Annealing
by: Li, Jiacheng, et al.
Published: (2025)

LoRA-Gen: Specializing Large Language Model via Online LoRA Generation
by: Xiao, Yicheng, et al.
Published: (2025)

LoRA-Squeeze: Simple and Effective Post-Tuning and In-Tuning Compression of LoRA Modules
by: Vulić, Ivan, et al.
Published: (2026)

LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization
by: Yen, Jui-Nan, et al.
Published: (2024)

NP-LoRA: Null Space Projection for Subject-Style LoRA Fusion
by: Chen, Chuheng, et al.
Published: (2025)

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
by: Li, Dengchun, et al.
Published: (2024)

Adaptive Selection of LoRA Components in Privacy-Preserving Federated Learning
by: Kim, Myoungjun, et al.
Published: (2026)

A Note on LoRA
by: Fomenko, Vlad, et al.
Published: (2024)

CoMoL: Efficient Mixture of LoRA Experts via Dynamic Core Space Merging
by: Cao, Jie, et al.
Published: (2026)

LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
by: Bałazy, Klaudia, et al.
Published: (2024)

Higher Layers Need More LoRA Experts
by: Gao, Chongyang, et al.
Published: (2024)

DMFI: A Dual-Modality Log Analysis Framework for Insider Threat Detection with LoRA-Tuned Language Models
by: Kong, Kaichuan, et al.
Published: (2025)

LoRA-FAIR: Federated LoRA Fine-Tuning with Aggregation and Initialization Refinement
by: Bian, Jieming, et al.
Published: (2024)

LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks
by: Prabhakar, Akshara, et al.
Published: (2024)

HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performance
by: Zhang, Hao, et al.
Published: (2025)

LoRATK: LoRA Once, Backdoor Everywhere in the Share-and-Play Ecosystem
by: Liu, Hongyi, et al.
Published: (2024)