Cover Image

Preventing Catastrophic Forgetting: Behavior-Aware Sampling for Safer Language Model Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pham, Anh, Thalanki, Mihir, Sun, Michael, Chaloo, Aditya, Gupta, Ankita, Xia, Tian, Mate, Aditya, Nosakhare, Ehimwenma, Srinivasan, Soundararajan
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting
by: Hancock, Asher J., et al.
Published: (2025)

Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting
by: Ding, Fei, et al.
Published: (2025)

Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgetting
by: Chen, Haolin, et al.
Published: (2024)

OPLoRA: Orthogonal Projection LoRA Prevents Catastrophic Forgetting during Parameter-Efficient Fine-Tuning
by: Xiong, Yifeng, et al.
Published: (2025)

Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning
by: Lou, Meng, et al.
Published: (2026)

Revisiting Catastrophic Forgetting in Large Language Model Tuning
by: Li, Hongyu, et al.
Published: (2024)

Robust Policy Optimization to Prevent Catastrophic Forgetting
by: Sabbaghi, Mahdi, et al.
Published: (2026)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
by: Fawi, Muhammad
Published: (2024)

Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning
by: Huo, Yujia, et al.
Published: (2025)

Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation
by: Bafghi, Reza Akbarian, et al.
Published: (2025)

On Catastrophic Forgetting in Low-Rank Decomposition-Based Parameter-Efficient Fine-Tuning
by: Ahmad, Muhammad, et al.
Published: (2026)

One Head, Many Models: Cross-Attention Routing for Cost-Aware LLM Selection
by: Pulishetty, Roshini, et al.
Published: (2025)

Preventing Catastrophic Forgetting through Memory Networks in Continuous Detection
by: Bhatt, Gaurav, et al.
Published: (2024)

Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning
by: Ren, Weijieying, et al.
Published: (2024)

G-Loss: Graph-Guided Fine-Tuning of Language Models
by: Sharma, Aditya, et al.
Published: (2026)

Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
by: Watts, Ishaan, et al.
Published: (2026)

Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting
by: Sanyal, Sunny, et al.
Published: (2025)

Model Editing at Scale leads to Gradual and Catastrophic Forgetting
by: Gupta, Akshat, et al.
Published: (2024)

Continual Deep Reinforcement Learning to Prevent Catastrophic Forgetting in Jamming Mitigation
by: Davaslioglu, Kemal, et al.
Published: (2024)

Keep the General, Inject the Specific: Structured Dialogue Fine-Tuning for Knowledge Injection without Catastrophic Forgetting
by: Hong, Yijie, et al.
Published: (2025)

Evolutionary Strategies lead to Catastrophic Forgetting in LLMs
by: Abdi, Immanuel, et al.
Published: (2026)

Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
by: Zhang, Zhixin, et al.
Published: (2025)

Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
by: Imanov, Olaf Yunus Laitinen
Published: (2026)

An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
by: Luo, Yun, et al.
Published: (2023)

Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
by: Huang, Wei, et al.
Published: (2025)

(How) Learning Rates Regulate Catastrophic Overtraining
by: Rofin, Mark, et al.
Published: (2026)

Enhancing Continual Learning for Software Vulnerability Prediction: Addressing Catastrophic Forgetting via Hybrid-Confidence-Aware Selective Replay for Temporal LLM Fine-Tuning
by: Dou, Xuhui, et al.
Published: (2026)

Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting
by: Liu, Chi, et al.
Published: (2026)

FedRef: Bayesian Fine-Tuning using a Reference Model to Mitigate Catastrophic Forgetting for Heterogeneous Federated Learning
by: Yoon, Taehwan, et al.
Published: (2025)

Continual Learning and Catastrophic Forgetting
by: van de Ven, Gido M., et al.
Published: (2024)

NUDGE: Lightweight Non-Parametric Fine-Tuning of Embeddings for Retrieval
by: Zeighami, Sepanta, et al.
Published: (2024)

Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning
by: Chen, Liang, et al.
Published: (2025)

Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategies
by: Koloski, Boshko, et al.
Published: (2023)

Balancing Speciality and Versatility: A Coarse to Fine Framework for Mitigating Catastrophic Forgetting in Large Language Models
by: Zhang, Hengyuan, et al.
Published: (2024)

Fine-Grained Gradient Restriction: A Simple Approach for Mitigating Catastrophic Forgetting
by: Liu, Bo, et al.
Published: (2024)

Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Models
by: Lu, Ke-Han, et al.
Published: (2025)

Transient Thermodynamic Efficiency of Adaptive Inference in Continuously Nonstationary Environments
by: Gupta, Aditya
Published: (2026)

An Efficient Sparse Fine-Tuning with Low Quantization Error via Neural Network Pruning
by: Li, Cen-Jhih, et al.
Published: (2025)

Solving the Catastrophic Forgetting Problem in Generalized Category Discovery
by: Cao, Xinzi, et al.
Published: (2025)

When Fine-Tuning Fails and when it Generalises: Role of Data Diversity and Mixed Training in LLM-based TTS
by: Purwar, Anupam, et al.
Published: (2026)