Cover Image

Mitigating Catastrophic Forgetting in Mathematical Reasoning Finetuning through Mixed Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Reynolds, John Graham
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
by: Watts, Ishaan, et al.
Published: (2026)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
by: Fawi, Muhammad
Published: (2024)

Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Models
by: Hsiao, Chi-Yuan, et al.
Published: (2025)

Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
by: Gupta, Prakhar, et al.
Published: (2026)

Understanding Catastrophic Forgetting in Language Models via Implicit Inference
by: Kotha, Suhas, et al.
Published: (2023)

Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
by: Zhang, Zhixin, et al.
Published: (2025)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
by: Bethune, Louis, et al.
Published: (2025)

Evolutionary Strategies lead to Catastrophic Forgetting in LLMs
by: Abdi, Immanuel, et al.
Published: (2026)

Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning
by: Ren, Weijieying, et al.
Published: (2024)

Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategies
by: Koloski, Boshko, et al.
Published: (2023)

Attractor Patch Networks: Reducing Catastrophic Forgetting with Routed Low-Rank Patch Experts
by: Shashank
Published: (2026)

Intelligent Learning Rate Distribution to reduce Catastrophic Forgetting in Transformers
by: Kenneweg, Philip, et al.
Published: (2024)

Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training
by: Lai, Song, et al.
Published: (2025)

Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
by: Chen, Howard, et al.
Published: (2025)

Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting
by: Liu, Chi, et al.
Published: (2026)

Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
by: Li, Junzhuo, et al.
Published: (2025)

Mitigating Catastrophic Forgetting in Continual Learning through Model Growth
by: Süalp, Ege, et al.
Published: (2025)

BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models
by: Chang, Yupeng, et al.
Published: (2024)

Randomized Masked Finetuning: An Efficient Way to Mitigate Memorization of PIIs in LLMs
by: Joshi, Kunj, et al.
Published: (2025)

Can A Gamer Train A Mathematical Reasoning Model?
by: Shin, Andrew
Published: (2025)

Flexi-LoRA with Input-Adaptive Ranks: Efficient Finetuning for Speech and Reasoning Tasks
by: Li, Zongqian, et al.
Published: (2026)

Flow-DPO: Improving LLM Mathematical Reasoning through Online Multi-Agent Learning
by: Deng, Yihe, et al.
Published: (2024)

Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains
by: Subramaniam, Vighnesh, et al.
Published: (2025)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
by: Zhang, Biao, et al.
Published: (2024)

Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
by: Vasudeva, Bhavya, et al.
Published: (2026)

Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting
by: Diao, Muxi, et al.
Published: (2026)

Time Sensitive Knowledge Editing through Efficient Finetuning
by: Ge, Xiou, et al.
Published: (2024)

Improving Sparse Memory Finetuning
by: Goyal, Satyam, et al.
Published: (2026)

Predicting Emergent Capabilities by Finetuning
by: Snell, Charlie, et al.
Published: (2024)

Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
by: Huang, Wei, et al.
Published: (2025)

Offline Learning and Forgetting for Reasoning with Large Language Models
by: Ni, Tianwei, et al.
Published: (2025)

Reasoning in Transformers -- Mitigating Spurious Correlations and Reasoning Shortcuts
by: Enström, Daniel, et al.
Published: (2024)

Mapping Post-Training Forgetting in Language Models at Scale
by: Harmon, Jackson, et al.
Published: (2025)

Catastrophic Forgetting Mitigation via Discrepancy-Weighted Experience Replay
by: Xu, Xinrun, et al.
Published: (2025)

Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
by: Shang, Bingqi, et al.
Published: (2025)

Reasoning Bias of Next Token Prediction Training
by: Lin, Pengxiao, et al.
Published: (2025)

Assessing the Portability of Parameter Matrices Trained by Parameter-Efficient Finetuning Methods
by: Sabry, Mohammed, et al.
Published: (2024)

Sparse Upcycling: Inference Inefficient Finetuning
by: Doubov, Sasha, et al.
Published: (2024)

Understanding Finetuning for Factual Knowledge Extraction
by: Ghosal, Gaurav, et al.
Published: (2024)

(How) Learning Rates Regulate Catastrophic Overtraining
by: Rofin, Mark, et al.
Published: (2026)