Image de couverture de livre

An Efficient Rehearsal Scheme for Catastrophic Forgetting Mitigation during Multi-stage Fine-tuning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Bai, Andrew, Yeh, Chih-Kuan, Hsieh, Cho-Jui, Taly, Ankur
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

On the Loss of Context-awareness in General Instruction Fine-tuning
par: Wang, Yihan, et autres
Publié: (2024)

Concept Gradient: Concept-based Interpretation Without Linear Assumption
par: Bai, Andrew, et autres
Publié: (2022)

Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
par: Zhang, Zhixin, et autres
Publié: (2025)

Two-stage LLM Fine-tuning with Less Specialization and More Generalization
par: Wang, Yihan, et autres
Publié: (2022)

Fine-Grained Gradient Restriction: A Simple Approach for Mitigating Catastrophic Forgetting
par: Liu, Bo, et autres
Publié: (2024)

Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
par: Huang, Wei, et autres
Publié: (2025)

Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgetting
par: Chen, Haolin, et autres
Publié: (2024)

Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
par: Watts, Ishaan, et autres
Publié: (2026)

Sequencing to Mitigate Catastrophic Forgetting in Continual Learning
par: Moussa, Hesham G., et autres
Publié: (2025)

On Catastrophic Forgetting in Low-Rank Decomposition-Based Parameter-Efficient Fine-Tuning
par: Ahmad, Muhammad, et autres
Publié: (2026)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
par: Fawi, Muhammad
Publié: (2024)

Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning
par: Huo, Yujia, et autres
Publié: (2025)

Concepts or Skills? Rethinking Instruction Selection for Multi-modal Models
par: Bai, Andrew, et autres
Publié: (2025)

Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problem
par: Wołczyk, Maciej, et autres
Publié: (2024)

Mitigating Catastrophic Forgetting in Language Transfer via Model Merging
par: Alexandrov, Anton, et autres
Publié: (2024)

Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation
par: Xie, Tong, et autres
Publié: (2024)

Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Models
par: Hsiao, Chi-Yuan, et autres
Publié: (2025)

MEGA: Second-Order Gradient Alignment for Catastrophic Forgetting Mitigation in GFSCIL
par: Pang, Jinhui, et autres
Publié: (2025)

Don't Forget the Critic: Value-Based Data Rehearsal for Multi-Cyclic Continual Reinforcement Learning
par: Poole, Benjamin, et autres
Publié: (2026)

Catastrophic Forgetting Mitigation Through Plateau Phase Activity Profiling
par: Mashiach, Idan, et autres
Publié: (2025)

Eidetic Learning: an Efficient and Provable Solution to Catastrophic Forgetting
par: Dronen, Nicholas, et autres
Publié: (2025)

Dealing with the Evil Twins: Improving Random Augmentation by Addressing Catastrophic Forgetting of Diverse Augmentations
par: Cho, Dongkyu, et autres
Publié: (2025)

Unlabeled Data Improves Fine-Grained Image Zero-shot Classification with Multimodal LLMs
par: Hong, Yunqi, et autres
Publié: (2025)

CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay
par: Zhang, Jianshu, et autres
Publié: (2024)

Mitigating Catastrophic Forgetting in Mathematical Reasoning Finetuning through Mixed Training
par: Reynolds, John Graham
Publié: (2025)

Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning
par: Chen, Liang, et autres
Publié: (2025)

Embedding Space Selection for Detecting Memorization and Fingerprinting in Generative Models
par: He, Jack, et autres
Publié: (2024)

Grow, Don't Overwrite: Fine-tuning Without Forgetting
par: Adila, Dyah, et autres
Publié: (2026)

FlexFed: Mitigating Catastrophic Forgetting in Heterogeneous Federated Learning in Pervasive Computing Environments
par: Alosaime, Sara, et autres
Publié: (2025)

Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
par: Imanov, Olaf Yunus Laitinen
Publié: (2026)

Predicting the Susceptibility of Examples to Catastrophic Forgetting
par: Hacohen, Guy, et autres
Publié: (2024)

Catastrophic Forgetting Mitigation via Discrepancy-Weighted Experience Replay
par: Xu, Xinrun, et autres
Publié: (2025)

Mitigating Bias in Dataset Distillation
par: Cui, Justin, et autres
Publié: (2024)

Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)
par: Kenthapadi, Krishnaram, et autres
Publié: (2024)

Efficient Frameworks for Generalized Low-Rank Matrix Bandit Problems
par: Kang, Yue, et autres
Publié: (2024)

Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning
par: Ren, Weijieying, et autres
Publié: (2024)

Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning
par: Lou, Meng, et autres
Publié: (2026)

Continual Deep Reinforcement Learning to Prevent Catastrophic Forgetting in Jamming Mitigation
par: Davaslioglu, Kemal, et autres
Publié: (2024)

CLUE: Concept-Level Uncertainty Estimation for Large Language Models
par: Wang, Yu-Hsiang, et autres
Publié: (2024)

Robust Policy Optimization to Prevent Catastrophic Forgetting
par: Sabbaghi, Mahdi, et autres
Publié: (2026)