Cover Image

An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Luo, Yun, Yang, Zhen, Meng, Fandong, Li, Yafu, Zhou, Jie, Zhang, Yue
Format:	Preprint
Published:	2023
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
by: Imanov, Olaf Yunus Laitinen
Published: (2026)

Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
by: Zhang, Zhixin, et al.
Published: (2025)

TEAL: Tokenize and Embed ALL for Multi-modal Large Language Models
by: Yang, Zhen, et al.
Published: (2023)

Revisiting Catastrophic Forgetting in Large Language Model Tuning
by: Li, Hongyu, et al.
Published: (2024)

Continuous Autoregressive Language Models
by: Shao, Chenze, et al.
Published: (2025)

SEE: Continual Fine-tuning with Sequential Ensemble of Experts
by: Wang, Zhilin, et al.
Published: (2025)

Balancing Speciality and Versatility: A Coarse to Fine Framework for Mitigating Catastrophic Forgetting in Large Language Models
by: Zhang, Hengyuan, et al.
Published: (2024)

An Empirical Study of Many-to-Many Summarization with Large Language Models
by: Wang, Jiaan, et al.
Published: (2025)

LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translation
by: Yin, Yongjing, et al.
Published: (2024)

Accelerating Inference in Large Language Models with a Unified Layer Skipping Strategy
by: Liu, Yijin, et al.
Published: (2024)

Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decoding
by: Zeng, Jiali, et al.
Published: (2023)

Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting
by: Ding, Fei, et al.
Published: (2025)

TIM: Teaching Large Language Models to Translate with Comparison
by: Zeng, Jiali, et al.
Published: (2023)

Beyond Next Token Prediction: Patch-Level Training for Large Language Models
by: Shao, Chenze, et al.
Published: (2024)

A Comparative Empirical Study of Catastrophic Forgetting Mitigation in Sequential Task Adaptation for Continual Natural Language Processing Systems
by: Abrahamyan, Aram, et al.
Published: (2026)

Large Language Models Are Not Robust Multiple Choice Selectors
by: Zheng, Chujie, et al.
Published: (2023)

XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners
by: Luo, Yun, et al.
Published: (2023)

Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models
by: Zhu, Didi, et al.
Published: (2024)

Fine-tuning a Large Language Model for Automating Computational Fluid Dynamics Simulations
by: Dong, Zhehao, et al.
Published: (2025)

Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal
by: Huang, Jianheng, et al.
Published: (2024)

CRAT: A Multi-Agent Framework for Causality-Enhanced Reflective and Retrieval-Augmented Translation with Large Language Models
by: Chen, Meiqi, et al.
Published: (2024)

Efficient Speech Language Modeling via Energy Distance in Continuous Latent Space
by: Ma, Zhengrui, et al.
Published: (2025)

Mitigating Catastrophic Forgetting in Continual Learning through Model Growth
by: Süalp, Ege, et al.
Published: (2025)

Preventing Catastrophic Forgetting: Behavior-Aware Sampling for Safer Language Model Fine-Tuning
by: Pham, Anh, et al.
Published: (2025)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
by: Fawi, Muhammad
Published: (2024)

Semi-supervised Fine-tuning for Large Language Models
by: Luo, Junyu, et al.
Published: (2024)

MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgetting
by: Li, Tianhao, et al.
Published: (2024)

EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translation
by: Xu, Yulin, et al.
Published: (2022)

Dissecting Long-Chain-of-Thought Reasoning Models: An Empirical Study
by: Mu, Yongyu, et al.
Published: (2025)

CM-Align: Consistency-based Multilingual Alignment for Large Language Models
by: Zhang, Xue, et al.
Published: (2025)

Generative Multi-Modal Knowledge Retrieval with Large Language Models
by: Long, Xinwei, et al.
Published: (2024)

TasTe: Teaching Large Language Models to Translate through Self-Reflection
by: Wang, Yutong, et al.
Published: (2024)

Countering Catastrophic Forgetting of Large Language Models for Better Instruction Following via Weight-Space Model Merging
by: Lyu, Mengxian, et al.
Published: (2026)

An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models
by: Zhou, Xiongtao, et al.
Published: (2024)

MiniPLM: Knowledge Distillation for Pre-Training Language Models
by: Gu, Yuxian, et al.
Published: (2024)

Understanding Catastrophic Forgetting in Language Models via Implicit Inference
by: Kotha, Suhas, et al.
Published: (2023)

Model-Dowser: Data-Free Importance Probing to Mitigate Catastrophic Forgetting in Multimodal Large Language Models
by: Hwang, Hyeontaek, et al.
Published: (2026)

Parameter Alignment Mitigates Catastrophic Forgetting in Multilingual Expert Language Models
by: Ahuja, Sanchit, et al.
Published: (2026)

EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning
by: Huang, Dong, et al.
Published: (2024)

Keys to Robust Edits: from Theoretical Insights to Practical Advances
by: Yan, Jianhao, et al.
Published: (2024)