Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.02129 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917662801526784 |
|---|---|
| author | Li, Zhoubo Zhang, Ningyu Yao, Yunzhi Wang, Mengru Chen, Xi Chen, Huajun |
| author_facet | Li, Zhoubo Zhang, Ningyu Yao, Yunzhi Wang, Mengru Chen, Xi Chen, Huajun |
| contents | As the cost associated with fine-tuning Large Language Models (LLMs) continues to rise, recent research efforts have pivoted towards developing methodologies to edit implicit knowledge embedded within LLMs. Yet, there's still a dark cloud lingering overhead -- will knowledge editing trigger butterfly effect? since it is still unclear whether knowledge editing might introduce side effects that pose potential risks or not. This paper pioneers the investigation into the potential pitfalls associated with knowledge editing for LLMs. To achieve this, we introduce new benchmark datasets and propose innovative evaluation metrics. Our results underline two pivotal concerns: (1) Knowledge Conflict: Editing groups of facts that logically clash can magnify the inherent inconsistencies in LLMs-a facet neglected by previous methods. (2) Knowledge Distortion: Altering parameters with the aim of editing factual knowledge can irrevocably warp the innate knowledge structure of LLMs. Experimental results vividly demonstrate that knowledge editing might inadvertently cast a shadow of unintended consequences on LLMs, which warrant attention and efforts for future works. Code and data are available at https://github.com/zjunlp/PitfallsKnowledgeEditing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_02129 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Unveiling the Pitfalls of Knowledge Editing for Large Language Models Li, Zhoubo Zhang, Ningyu Yao, Yunzhi Wang, Mengru Chen, Xi Chen, Huajun Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Databases Machine Learning As the cost associated with fine-tuning Large Language Models (LLMs) continues to rise, recent research efforts have pivoted towards developing methodologies to edit implicit knowledge embedded within LLMs. Yet, there's still a dark cloud lingering overhead -- will knowledge editing trigger butterfly effect? since it is still unclear whether knowledge editing might introduce side effects that pose potential risks or not. This paper pioneers the investigation into the potential pitfalls associated with knowledge editing for LLMs. To achieve this, we introduce new benchmark datasets and propose innovative evaluation metrics. Our results underline two pivotal concerns: (1) Knowledge Conflict: Editing groups of facts that logically clash can magnify the inherent inconsistencies in LLMs-a facet neglected by previous methods. (2) Knowledge Distortion: Altering parameters with the aim of editing factual knowledge can irrevocably warp the innate knowledge structure of LLMs. Experimental results vividly demonstrate that knowledge editing might inadvertently cast a shadow of unintended consequences on LLMs, which warrant attention and efforts for future works. Code and data are available at https://github.com/zjunlp/PitfallsKnowledgeEditing. |
| title | Unveiling the Pitfalls of Knowledge Editing for Large Language Models |
| topic | Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Databases Machine Learning |
| url | https://arxiv.org/abs/2310.02129 |