Cover Image

Understanding Catastrophic Forgetting in Language Models via Implicit Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kotha, Suhas, Springer, Jacob Mitchell, Raghunathan, Aditi
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Repetition Improves Language Model Embeddings
by: Springer, Jacob Mitchell, et al.
Published: (2024)

Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
by: Watts, Ishaan, et al.
Published: (2026)

Testing the Limits of Jailbreaking Defenses with the Purple Problem
by: Kim, Taeyoun, et al.
Published: (2024)

Mitigating Bias in RAG: Controlling the Embedder
by: Kim, Taeyoun, et al.
Published: (2025)

Replaying pre-training data improves fine-tuning
by: Kotha, Suhas, et al.
Published: (2026)

Understanding Finetuning for Factual Knowledge Extraction
by: Ghosal, Gaurav, et al.
Published: (2024)

Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learning
by: Springer, Jacob Mitchell, et al.
Published: (2024)

Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs
by: Zhong, Ziqian, et al.
Published: (2025)

Understanding the Influence of Synthetic Data for Text Embedders
by: Springer, Jacob Mitchell, et al.
Published: (2025)

Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models
by: Wang, Huazheng, et al.
Published: (2025)

ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases
by: Zhong, Ziqian, et al.
Published: (2025)

Evolutionary Strategies lead to Catastrophic Forgetting in LLMs
by: Abdi, Immanuel, et al.
Published: (2026)

Self-Trained Verification for Training- and Test-Time Self-Improvement
by: Wu, Chen Henry, et al.
Published: (2026)

Mitigating Catastrophic Forgetting in Mathematical Reasoning Finetuning through Mixed Training
by: Reynolds, John Graham
Published: (2025)

Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning
by: Ren, Weijieying, et al.
Published: (2024)

Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategies
by: Koloski, Boshko, et al.
Published: (2023)

Attractor Patch Networks: Reducing Catastrophic Forgetting with Routed Low-Rank Patch Experts
by: Shashank
Published: (2026)

Intelligent Learning Rate Distribution to reduce Catastrophic Forgetting in Transformers
by: Kenneweg, Philip, et al.
Published: (2024)

Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliance
by: Goyal, Sachin, et al.
Published: (2024)

Mode-Conditioning Unlocks Superior Test-Time Scaling
by: Wu, Chen Henry, et al.
Published: (2025)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
by: Fawi, Muhammad
Published: (2024)

Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Models
by: Hsiao, Chi-Yuan, et al.
Published: (2025)

Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting
by: Liu, Chi, et al.
Published: (2026)

Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
by: Li, Junzhuo, et al.
Published: (2025)

Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction
by: Nagarajan, Vaishnavh, et al.
Published: (2025)

Learning and Forgetting Unsafe Examples in Large Language Models
by: Zhao, Jiachen, et al.
Published: (2023)

Base Models Look Human To AI Detectors
by: Xu, Yixuan Even, et al.
Published: (2026)

Demystifying Language Model Forgetting with Low-rank Example Associations
by: Jin, Xisen, et al.
Published: (2024)

Early Data Exposure Improves Robustness to Subsequent Fine-Tuning
by: Feng, Lawrence, et al.
Published: (2026)

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
by: Sardana, Nikhil, et al.
Published: (2023)

Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
by: Zhang, Zhixin, et al.
Published: (2025)

Graceful Forgetting in Generative Language Models
by: Jiang, Chunyang, et al.
Published: (2025)

What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinement
by: Jin, Xisen, et al.
Published: (2024)

Elephants Never Forget: Testing Language Models for Memorization of Tabular Data
by: Bordt, Sebastian, et al.
Published: (2024)

InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Models
by: Gu, Yanggan, et al.
Published: (2025)

Improving Language Plasticity via Pretraining with Active Forgetting
by: Chen, Yihong, et al.
Published: (2023)

Overtrained Language Models Are Harder to Fine-Tune
by: Springer, Jacob Mitchell, et al.
Published: (2025)

Bootstrapping Language Models with DPO Implicit Rewards
by: Chen, Changyu, et al.
Published: (2024)

Discovering Implicit Large Language Model Alignment Objectives
by: Chen, Edward, et al.
Published: (2026)

Reframing Data Value for Large Language Models Through the Lens of Plausibility
by: Rammal, Mohamad Rida, et al.
Published: (2024)