Cover Image

Low-Rank Adaptation Reduces Catastrophic Forgetting in Sequential Transformer Encoder Fine-Tuning: Controlled Empirical Evidence and Frozen-Backbone Representation Probes

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Pandey, Ashish
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On Catastrophic Forgetting in Low-Rank Decomposition-Based Parameter-Efficient Fine-Tuning
by: Ahmad, Muhammad, et al.
Published: (2026)

Subspace Geometry Governs Catastrophic Forgetting in Low-Rank Adaptation
by: Steele, Brady
Published: (2026)

Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation
by: Bafghi, Reza Akbarian, et al.
Published: (2025)

Attractor Patch Networks: Reducing Catastrophic Forgetting with Routed Low-Rank Patch Experts
by: Shashank
Published: (2026)

Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning
by: Ren, Weijieying, et al.
Published: (2024)

Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning
by: Huo, Yujia, et al.
Published: (2025)

Mitigating Forgetting in Low Rank Adaptation
by: Sliwa, Joanna, et al.
Published: (2025)

Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgetting
by: Chen, Haolin, et al.
Published: (2024)

Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning
by: Lou, Meng, et al.
Published: (2026)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
by: Fawi, Muhammad
Published: (2024)

Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation
by: Gu, Hao, et al.
Published: (2026)

Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models
by: Hu, Jerry Yao-Chieh, et al.
Published: (2024)

Avoid Catastrophic Forgetting with Rank-1 Fisher from Diffusion Models
by: Wang, Zekun, et al.
Published: (2025)

AdapterTune: Zero-Initialized Low-Rank Adapters for Frozen Vision Transformers
by: Khazem, Salim
Published: (2026)

Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
by: Raje, Arian, et al.
Published: (2025)

Transfer Learning in Physics-Informed Neural Networks: Full Fine-Tuning, Lightweight Fine-Tuning, and Low-Rank Adaptation
by: Wang, Yizheng, et al.
Published: (2025)

ScaLoRA: Optimally Scaled Low-Rank Adaptation for Efficient High-Rank Fine-Tuning
by: Zhang, Yilang, et al.
Published: (2025)

An Empirical Analysis of Forgetting in Pre-trained Models with Incremental Low-Rank Updates
by: Soutif--Cormerais, Albin, et al.
Published: (2024)

NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Models
by: Lin, Cheng, et al.
Published: (2024)

Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions
by: Zhang, Wenhao, et al.
Published: (2026)

AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning
by: Zhang, Ruiyi, et al.
Published: (2024)

Optimizing Fine-Tuning through Advanced Initialization Strategies for Low-Rank Adaptation
by: Xue, Yongfu
Published: (2025)

LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning
by: Tastan, Nurbek, et al.
Published: (2025)

Fine-Grained Gradient Restriction: A Simple Approach for Mitigating Catastrophic Forgetting
by: Liu, Bo, et al.
Published: (2024)

RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models
by: Zhang, Yilang, et al.
Published: (2025)

Predicting the Susceptibility of Examples to Catastrophic Forgetting
by: Hacohen, Guy, et al.
Published: (2024)

Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning
by: Fujisawa, Yoshihiko, et al.
Published: (2026)

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
by: Zhang, Haojie
Published: (2025)

GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
by: Jung, Yeonjoon, et al.
Published: (2025)

Sequential Learning and Catastrophic Forgetting in Differentiable Resistor Networks
by: Ibrahim, Maniru
Published: (2026)

An Efficient Rehearsal Scheme for Catastrophic Forgetting Mitigation during Multi-stage Fine-tuning
by: Bai, Andrew, et al.
Published: (2024)

Multi-Head Attention as a Source of Catastrophic Forgetting in MoE Transformers
by: Chen, Anrui, et al.
Published: (2026)

Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategies
by: Koloski, Boshko, et al.
Published: (2023)

Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
by: Huang, Wei, et al.
Published: (2025)

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
by: Zhang, Chengqian, et al.
Published: (2026)

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
by: Zhang, Longteng, et al.
Published: (2026)

Robust Policy Optimization to Prevent Catastrophic Forgetting
by: Sabbaghi, Mahdi, et al.
Published: (2026)

Intelligent Learning Rate Distribution to reduce Catastrophic Forgetting in Transformers
by: Kenneweg, Philip, et al.
Published: (2024)

How to Leverage Predictive Uncertainty Estimates for Reducing Catastrophic Forgetting in Online Continual Learning
by: Serra, Giuseppe, et al.
Published: (2024)

QR-LoRA: QR-Based Low-Rank Adaptation for Efficient Fine-Tuning of Large Language Models
by: Liang, Jessica, et al.
Published: (2025)