Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editing
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909625761136640 |
|---|---|
| author | Liu, Tianci Li, Ruirui Dong, Zihan Liu, Hui Tang, Xianfeng Yin, Qingyu Zhang, Linjun Wang, Haoyu Gao, Jing |
| author_facet | Liu, Tianci Li, Ruirui Dong, Zihan Liu, Hui Tang, Xianfeng Yin, Qingyu Zhang, Linjun Wang, Haoyu Gao, Jing |
| contents | Large language models (LLMs) have achieved remarkable performance on various natural language tasks. However, they are trained on static corpora and their knowledge can become outdated quickly in the fast-changing world. This motivates the development of knowledge editing (KE) to update specific knowledge in LLMs without changing unrelated others or compromising their pre-trained capabilities. Previous efforts sought to update a small amount of parameters of a LLM and proved effective for making selective updates. Nonetheless, the edited LLM often exhibits degraded ability to reason about the new knowledge. In this work, we identify a key issue: heterogeneous token overfitting (HTO), where the LLM overfits different tokens in the provided knowledge at varying rates. To tackle this, we propose OVERTONE, a token-level smoothing method that mitigates HTO by adaptively refining the target distribution. Theoretically, OVERTONE offers better parameter updates with negligible computation overhead. It also induces an implicit DPO but does not require preference data pairs. Extensive experiments across four editing methods, two LLMs, and diverse scenarios demonstrate the effectiveness and versatility of our method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_00602 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editing Liu, Tianci Li, Ruirui Dong, Zihan Liu, Hui Tang, Xianfeng Yin, Qingyu Zhang, Linjun Wang, Haoyu Gao, Jing Computation and Language Machine Learning Large language models (LLMs) have achieved remarkable performance on various natural language tasks. However, they are trained on static corpora and their knowledge can become outdated quickly in the fast-changing world. This motivates the development of knowledge editing (KE) to update specific knowledge in LLMs without changing unrelated others or compromising their pre-trained capabilities. Previous efforts sought to update a small amount of parameters of a LLM and proved effective for making selective updates. Nonetheless, the edited LLM often exhibits degraded ability to reason about the new knowledge. In this work, we identify a key issue: heterogeneous token overfitting (HTO), where the LLM overfits different tokens in the provided knowledge at varying rates. To tackle this, we propose OVERTONE, a token-level smoothing method that mitigates HTO by adaptively refining the target distribution. Theoretically, OVERTONE offers better parameter updates with negligible computation overhead. It also induces an implicit DPO but does not require preference data pairs. Extensive experiments across four editing methods, two LLMs, and diverse scenarios demonstrate the effectiveness and versatility of our method. |
| title | Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editing |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2502.00602 |