WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916274149261312 |
|---|---|
| author | Hu, Chenhui Cao, Pengfei Chen, Yubo Liu, Kang Zhao, Jun |
| author_facet | Hu, Chenhui Cao, Pengfei Chen, Yubo Liu, Kang Zhao, Jun |
| contents | Knowledge editing aims to rectify inaccuracies in large language models (LLMs) without costly retraining for outdated or erroneous knowledge. However, current knowledge editing methods primarily focus on single editing, failing to meet the requirements for lifelong editing. This study reveals a performance degradation encountered by knowledge editing in lifelong editing, characterized by toxicity buildup and toxicity flash, with the primary cause identified as pattern unmatch. We introduce a knowledge editing approach named Wise-Layer Knowledge Editor (WilKE), which selects editing layer based on the pattern matching degree of editing knowledge across different layers in language models. Experimental results demonstrate that, in lifelong editing, WilKE exhibits an average improvement of 46.2% and 67.8% on editing GPT2-XL and GPT-J relative to state-of-the-art knowledge editing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_10987 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing Hu, Chenhui Cao, Pengfei Chen, Yubo Liu, Kang Zhao, Jun Computation and Language Artificial Intelligence Knowledge editing aims to rectify inaccuracies in large language models (LLMs) without costly retraining for outdated or erroneous knowledge. However, current knowledge editing methods primarily focus on single editing, failing to meet the requirements for lifelong editing. This study reveals a performance degradation encountered by knowledge editing in lifelong editing, characterized by toxicity buildup and toxicity flash, with the primary cause identified as pattern unmatch. We introduce a knowledge editing approach named Wise-Layer Knowledge Editor (WilKE), which selects editing layer based on the pattern matching degree of editing knowledge across different layers in language models. Experimental results demonstrate that, in lifelong editing, WilKE exhibits an average improvement of 46.2% and 67.8% on editing GPT2-XL and GPT-J relative to state-of-the-art knowledge editing methods. |
| title | WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2402.10987 |