Cover Image

On Catastrophic Forgetting in Low-Rank Decomposition-Based Parameter-Efficient Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ahmad, Muhammad, Zheng, Jingjing, Cao, Yankai
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Adaptive Principal Components Allocation with the $\ell_{2,g}$-regularized Gaussian Graphical Model for Efficient Fine-Tuning Large Models
by: Zheng, Jingjing, et al.
Published: (2024)

Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgetting
by: Chen, Haolin, et al.
Published: (2024)

Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning
by: Ren, Weijieying, et al.
Published: (2024)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
by: Fawi, Muhammad
Published: (2024)

Low-Rank Adaptation Reduces Catastrophic Forgetting in Sequential Transformer Encoder Fine-Tuning: Controlled Empirical Evidence and Frozen-Backbone Representation Probes
by: Pandey, Ashish
Published: (2026)

Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning
by: Lou, Meng, et al.
Published: (2026)

Subspace Geometry Governs Catastrophic Forgetting in Low-Rank Adaptation
by: Steele, Brady
Published: (2026)

CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay
by: Zhang, Jianshu, et al.
Published: (2024)

FuRA: Full-Rank Parameter-Efficient Fine-Tuning with Spectral Preconditioning
by: Zhao, Yequan, et al.
Published: (2026)

Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning
by: Huo, Yujia, et al.
Published: (2025)

Attractor Patch Networks: Reducing Catastrophic Forgetting with Routed Low-Rank Patch Experts
by: Shashank
Published: (2026)

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
by: Zhang, Haojie
Published: (2025)

GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
by: Jung, Yeonjoon, et al.
Published: (2025)

An Efficient Rehearsal Scheme for Catastrophic Forgetting Mitigation during Multi-stage Fine-tuning
by: Bai, Andrew, et al.
Published: (2024)

Avoid Catastrophic Forgetting with Rank-1 Fisher from Diffusion Models
by: Wang, Zekun, et al.
Published: (2025)

LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning
by: Zhang, Mingyang, et al.
Published: (2023)

Eidetic Learning: an Efficient and Provable Solution to Catastrophic Forgetting
by: Dronen, Nicholas, et al.
Published: (2025)

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
by: Jiang, Ting, et al.
Published: (2024)

Fine-Grained Gradient Restriction: A Simple Approach for Mitigating Catastrophic Forgetting
by: Liu, Bo, et al.
Published: (2024)

ARD-LoRA: Dynamic Rank Allocation for Parameter-Efficient Fine-Tuning of Foundation Models with Heterogeneous Adaptation Needs
by: Shinwari, Haseeb Ullah Khan, et al.
Published: (2025)

Predicting the Susceptibility of Examples to Catastrophic Forgetting
by: Hacohen, Guy, et al.
Published: (2024)

AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models
by: Liu, Zeyu, et al.
Published: (2024)

Don't Forget the Nonlinearity: Unlocking Activation Functions in Efficient Fine-Tuning
by: Yin, Bo, et al.
Published: (2025)

ScaLoRA: Optimally Scaled Low-Rank Adaptation for Efficient High-Rank Fine-Tuning
by: Zhang, Yilang, et al.
Published: (2025)

Mitigating Forgetting in Low Rank Adaptation
by: Sliwa, Joanna, et al.
Published: (2025)

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression
by: Shi, Jiang-Xin, et al.
Published: (2025)

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
by: Zhang, Longteng, et al.
Published: (2026)

Catastrophic Forgetting Resilient One-Shot Incremental Federated Learning
by: Zaland, Obaidullah, et al.
Published: (2026)

Task-Adaptive Parameter-Efficient Fine-Tuning for Weather Foundation Models
by: Cao, Shilei, et al.
Published: (2025)

Graph-Based Spectral Decomposition for Parameter Coordination in Language Model Fine-Tuning
by: Zhang, Hanlu, et al.
Published: (2025)

Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategies
by: Koloski, Boshko, et al.
Published: (2023)

Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
by: Huang, Wei, et al.
Published: (2025)

NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Models
by: Lin, Cheng, et al.
Published: (2024)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
by: Yang, Yifan, et al.
Published: (2024)

QR-LoRA: QR-Based Low-Rank Adaptation for Efficient Fine-Tuning of Large Language Models
by: Liang, Jessica, et al.
Published: (2025)

Robust Policy Optimization to Prevent Catastrophic Forgetting
by: Sabbaghi, Mahdi, et al.
Published: (2026)

Catastrophic Forgetting in Kolmogorov-Arnold Networks
by: Rahman, Mohammad Marufur, et al.
Published: (2025)

Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning
by: Jiang, Gangwei, et al.
Published: (2025)

LoCA: Location-Aware Cosine Adaptation for Parameter-Efficient Fine-Tuning
by: Du, Zhekai, et al.
Published: (2025)

Decentralized Low-Rank Fine-Tuning of Large Language Models
by: Ghiasvand, Sajjad, et al.
Published: (2025)