Cover Image

Mitigating Forgetting in LLM Fine-Tuning via Low-Perplexity Token Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wu, Chao-Chung, Tam, Zhi Rui, Lin, Chieh-Yen, Chen, Yun-Nung, Sun, Shao-Hua, Lee, Hung-yi
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

I Need Help! Evaluating LLM's Ability to Ask for Users' Support: A Case Study on Text-to-SQL Generation
by: Wu, Cheng-Kuang, et al.
Published: (2024)

StreamBench: Towards Benchmarking Continuous Improvement of Language Agents
by: Wu, Cheng-Kuang, et al.
Published: (2024)

Answer, Refuse, or Guess? Investigating Risk-Aware Decision Making in Language Models
by: Wu, Cheng-Kuang, et al.
Published: (2025)

Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models
by: Tam, Zhi Rui, et al.
Published: (2024)

Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models?
by: Tam, Zhi Rui, et al.
Published: (2025)

On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluation
by: Hsu, Chan-Jan, et al.
Published: (2026)

On Calibration of Large Language Models: From Response To Capability
by: Yang, Sin-Han, et al.
Published: (2026)

None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answering
by: Tam, Zhi Rui, et al.
Published: (2025)

MedVoiceBias: A Controlled Study of Audio LLM Behavior in Clinical Decision-Making
by: Tam, Zhi Rui, et al.
Published: (2025)

VisTW: Benchmarking Vision-Language Models for Traditional Chinese in Taiwan
by: Tam, Zhi Rui, et al.
Published: (2025)

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
by: Chen, Yupeng, et al.
Published: (2024)

Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs
by: Chen, Yen-Shan, et al.
Published: (2026)

Transferring Textual Preferences to Vision-Language Understanding through Model Merging
by: Li, Chen-An, et al.
Published: (2025)

DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging
by: Lin, Tzu-Han, et al.
Published: (2024)

PRISM: A Geometric Risk Bound that Decomposes Drift into Scale, Shape, and Head
by: Lin, Chieh-Yen, et al.
Published: (2026)

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
by: Fawi, Muhammad
Published: (2024)

AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning
by: Lin, Tzu-Han, et al.
Published: (2025)

Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting
by: Sanyal, Sunny, et al.
Published: (2025)

Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging
by: Farn, Hua, et al.
Published: (2024)

Task Arithmetic can Mitigate Synthetic-to-Real Gap in Automatic Speech Recognition
by: Su, Hsuan, et al.
Published: (2024)

Creativity in LLM-based Multi-Agent Systems: A Survey
by: Lin, Yi-Cheng, et al.
Published: (2025)

Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training
by: Lai, Song, et al.
Published: (2025)

LLM-Codec: Neural Audio Codec Meets Language Model Objectives
by: Chung, Ho-Lam, et al.
Published: (2026)

Speech-FT: Merging Pre-trained And Fine-Tuned Speech Representation Models For Cross-Task Generalization
by: Lin, Tzu-Quan, et al.
Published: (2025)

LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play
by: Lu, Li-Chun, et al.
Published: (2024)

Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies
by: Schweighofer, Kajetan, et al.
Published: (2026)

Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning
by: Chen, Liang, et al.
Published: (2025)

Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting
by: Diao, Muxi, et al.
Published: (2026)

TRACT: Regression-Aware Fine-tuning Meets Chain-of-Thought Reasoning for LLM-as-a-Judge
by: Chiang, Cheng-Han, et al.
Published: (2025)

Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation
by: Bafghi, Reza Akbarian, et al.
Published: (2025)

Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Models
by: Hsiao, Chi-Yuan, et al.
Published: (2025)

TASTE-Streaming: Towards Streamable Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
by: Tseng, Liang-Hsuan, et al.
Published: (2026)

Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning
by: Huo, Yujia, et al.
Published: (2025)

Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting
by: Ding, Fei, et al.
Published: (2025)

Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Models
by: Lu, Ke-Han, et al.
Published: (2025)

Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation
by: Chen, Mu-Chi, et al.
Published: (2026)

Teaching Audio-Aware Large Language Models What Does Not Hear: Mitigating Hallucinations through Synthesized Negative Samples
by: Kuan, Chun-Yi, et al.
Published: (2025)

Style Amnesia: Investigating Speaking Style Degradation and Mitigation in Multi-Turn Spoken Language Models
by: Lin, Yu-Xiang, et al.
Published: (2025)

Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
by: Zhang, Zhixin, et al.
Published: (2025)

TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories
by: Chen, Yen-Shan, et al.
Published: (2026)