Cover Image

Revisiting Catastrophic Forgetting in Large Language Model Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Hongyu, Ding, Liang, Fang, Meng, Tao, Dacheng
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting
by: Ding, Fei, et al.
Published: (2025)

Preventing Catastrophic Forgetting: Behavior-Aware Sampling for Safer Language Model Fine-Tuning
by: Pham, Anh, et al.
Published: (2025)

Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal
by: Huang, Jianheng, et al.
Published: (2024)

Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning
by: Ren, Weijieying, et al.
Published: (2024)

MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgetting
by: Li, Tianhao, et al.
Published: (2024)

Countering Catastrophic Forgetting of Large Language Models for Better Instruction Following via Weight-Space Model Merging
by: Lyu, Mengxian, et al.
Published: (2026)

Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuning
by: Zan, Changtong, et al.
Published: (2024)

Keep the General, Inject the Specific: Structured Dialogue Fine-Tuning for Knowledge Injection without Catastrophic Forgetting
by: Hong, Yijie, et al.
Published: (2025)

Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning
by: Huang, Wenke, et al.
Published: (2024)

Improving Large Language Models with Concept-Aware Fine-Tuning
by: Chen, Michael K., et al.
Published: (2025)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
by: Fawi, Muhammad
Published: (2024)

Model Editing at Scale leads to Gradual and Catastrophic Forgetting
by: Gupta, Akshat, et al.
Published: (2024)

Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments
by: Lu, Qingyu, et al.
Published: (2025)

Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updates
by: Yamaguchi, Atsuki, et al.
Published: (2025)

NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Models
by: Ho, Zheng Yi, et al.
Published: (2024)

Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer
by: Liu, Boan, et al.
Published: (2023)

Edit Once, Update Everywhere: A Simple Framework for Cross-Lingual Knowledge Synchronization in LLMs
by: Wu, Yuchen, et al.
Published: (2025)

Evolutionary Strategies lead to Catastrophic Forgetting in LLMs
by: Abdi, Immanuel, et al.
Published: (2026)

Aligning Large Language Models from Self-Reference AI Feedback with one General Principle
by: Bao, Rong, et al.
Published: (2024)

Output Scouting: Auditing Large Language Models for Catastrophic Responses
by: Bell, Andrew, et al.
Published: (2024)

Large Language Models as an Indirect Reasoner: Contrapositive and Contradiction for Automated Reasoning
by: Zhang, Yanfang, et al.
Published: (2024)

Can Similarity-Based Domain-Ordering Reduce Catastrophic Forgetting for Intent Recognition?
by: Mannekote, Amogh, et al.
Published: (2024)

Retentive or Forgetful? Diving into the Knowledge Memorizing Mechanism of Language Models
by: Cao, Boxi, et al.
Published: (2023)

Talking to Yourself: Defying Forgetting in Large Language Models
by: Sun, Yutao, et al.
Published: (2026)

How to Alleviate Catastrophic Forgetting in LLMs Finetuning? Hierarchical Layer-Wise and Element-Wise Regularization
by: Song, Shezheng, et al.
Published: (2025)

SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignment
by: Huang, Yuqing, et al.
Published: (2025)

SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data
by: Fang, Wenkai, et al.
Published: (2025)

A Survey on Self-Evolution of Large Language Models
by: Tao, Zhengwei, et al.
Published: (2024)

Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
by: Zhang, Zhixin, et al.
Published: (2025)

Intelligent Learning Rate Distribution to reduce Catastrophic Forgetting in Transformers
by: Kenneweg, Philip, et al.
Published: (2024)

Unveiling and Addressing Pseudo Forgetting in Large Language Models
by: Sun, Huashan, et al.
Published: (2024)

Revisiting Large Language Model Pruning using Neuron Semantic Attribution
by: Ding, Yizhuo, et al.
Published: (2025)

Fast-Slow-Thinking: Complex Task Solving with Large Language Models
by: Sun, Yiliu, et al.
Published: (2025)

Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model
by: Huang, Wenke, et al.
Published: (2025)

Revisiting Uncertainty Estimation and Calibration of Large Language Models
by: Tao, Linwei, et al.
Published: (2025)

Joint Flashback Adaptation for Forgetting-Resistant Instruction Tuning
by: Zhao, Yukun, et al.
Published: (2025)

HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models
by: Dai, Chang, et al.
Published: (2025)

Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
by: Li, Junzhuo, et al.
Published: (2025)

Tuning Language Models for Robust Prediction of Diverse User Behaviors
by: Meng, Fanjin, et al.
Published: (2025)

Instruction Tuning Vs. In-Context Learning: Revisiting Large Language Models in Few-Shot Computational Social Science
by: Wang, Taihang, et al.
Published: (2024)