Cover Image

Forgetting: A New Mechanism Towards Better Large Language Model Fine-tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Taheri, Ali, Taban, Alireza, Wang, Qizhou, Ye, Shanshan, Mirzaei, Abdolreza, Liu, Tongliang, Han, Bo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Is Gradient Ascent Really Necessary? Memorize to Forget for Machine Unlearning
by: Huang, Zhuo, et al.
Published: (2026)

Towards Understanding Valuable Preference Data for Large Language Model Alignment
by: Zhang, Zizhuo, et al.
Published: (2025)

Towards Effective Evaluations and Comparisons for LLM Unlearning Methods
by: Wang, Qizhou, et al.
Published: (2024)

Least but not Last: Fine-tuning Intermediate Principal Components for Better Performance-Forgetting Trade-Offs
by: Quercia, Alessio, et al.
Published: (2026)

Knowledge Distillation on Spatial-Temporal Graph Convolutional Network for Traffic Prediction
by: Izadi, Mohammad, et al.
Published: (2024)

On the Thinking-Language Modeling Gap in Large Language Models
by: Liu, Chenxi, et al.
Published: (2025)

BarrierBench: Evaluating Large Language Models for Safety Verification in Dynamical Systems
by: Taheri, Ali, et al.
Published: (2025)

Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection
by: Cao, Chentao, et al.
Published: (2024)

Grow, Don't Overwrite: Fine-tuning Without Forgetting
by: Adila, Dyah, et al.
Published: (2026)

Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning
by: Yang, Puning, et al.
Published: (2025)

Fine-Grained Gradient Restriction: A Simple Approach for Mitigating Catastrophic Forgetting
by: Liu, Bo, et al.
Published: (2024)

Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problem
by: Wołczyk, Maciej, et al.
Published: (2024)

DeepInception: Hypnotize Large Language Model to Be Jailbreaker
by: Li, Xuan, et al.
Published: (2023)

Fine-tuning Large Language Model for Automated Algorithm Design
by: Liu, Fei, et al.
Published: (2025)

A Study of Optimizations for Fine-tuning Large Language Models
by: Singh, Arjun, et al.
Published: (2024)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
by: Song, Weixi, et al.
Published: (2023)

Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation
by: Huang, Kai, et al.
Published: (2023)

Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
by: Imanov, Olaf Yunus Laitinen
Published: (2026)

Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models
by: Yao, Kai, et al.
Published: (2024)

Discovering and Reasoning of Causality in the Hidden World with Large Language Models
by: Liu, Chenxi, et al.
Published: (2024)

Distinguishable Deletion: Unifying Knowledge Erasure and Refusal for Large Language Model Unlearning
by: Yang, Puning, et al.
Published: (2026)

Fine-tuning Large Language Models for Domain-specific Machine Translation
by: Zheng, Jiawei, et al.
Published: (2024)

TuneComp: Joint Fine-tuning and Compression for Large Foundation Models
by: Chen, Xiangyu, et al.
Published: (2025)

Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning
by: Huang, Wei, et al.
Published: (2025)

An Efficient Rehearsal Scheme for Catastrophic Forgetting Mitigation during Multi-stage Fine-tuning
by: Bai, Andrew, et al.
Published: (2024)

Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models
by: Chen, Pin-Yu, et al.
Published: (2025)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
by: Chen, Yukang, et al.
Published: (2023)

Exploring Memorization in Fine-tuned Language Models
by: Zeng, Shenglai, et al.
Published: (2023)

EDCO: Dynamic Curriculum Orchestration for Domain-specific Large Language Model Fine-tuning
by: Pang, Jing-Cheng, et al.
Published: (2026)

Noisy Test-Time Adaptation in Vision-Language Models
by: Cao, Chentao, et al.
Published: (2025)

Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attack
by: Huang, Tiansheng, et al.
Published: (2024)

Can Large Language Models Help Experimental Design for Causal Discovery?
by: Li, Junyi, et al.
Published: (2025)

Towards Benchmarking Privacy Vulnerabilities in Selective Forgetting with Large Language Models
by: Qian, Wei, et al.
Published: (2025)

Beyond Accuracy: On the Effects of Fine-tuning Towards Vision-Language Model's Prediction Rationality
by: Wang, Qitong, et al.
Published: (2024)

Towards Compatible Fine-tuning for Vision-Language Model Updates
by: Wang, Zhengbo, et al.
Published: (2024)

Scaling Laws for Forgetting When Fine-Tuning Large Language Models
by: Kalajdzievski, Damjan
Published: (2024)

Selecting Large Language Model to Fine-tune via Rectified Scaling Law
by: Lin, Haowei, et al.
Published: (2024)

Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
by: Chi, Haoang, et al.
Published: (2025)

Navigating the Designs of Privacy-Preserving Fine-tuning for Large Language Models
by: Shi, Haonan, et al.
Published: (2025)

Fine-tuning of Large Language Models for Domain-Specific Cybersecurity Knowledge
by: Huang, Yuan
Published: (2025)