Editing Conceptual Knowledge for Large Language Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929528923750400 |
|---|---|
| author | Wang, Xiaohan Mao, Shengyu Zhang, Ningyu Deng, Shumin Yao, Yunzhi Shen, Yue Liang, Lei Gu, Jinjie Chen, Huajun |
| author_facet | Wang, Xiaohan Mao, Shengyu Zhang, Ningyu Deng, Shumin Yao, Yunzhi Shen, Yue Liang, Lei Gu, Jinjie Chen, Huajun |
| contents | Recently, there has been a growing interest in knowledge editing for Large Language Models (LLMs). Current approaches and evaluations merely explore the instance-level editing, while whether LLMs possess the capability to modify concepts remains unclear. This paper pioneers the investigation of editing conceptual knowledge for LLMs, by constructing a novel benchmark dataset ConceptEdit and establishing a suite of new metrics for evaluation. The experimental results reveal that, although existing editing methods can efficiently modify concept-level definition to some extent, they also have the potential to distort the related instantial knowledge in LLMs, leading to poor performance. We anticipate this can inspire further progress in better understanding LLMs. Our project homepage is available at https://zjunlp.github.io/project/ConceptEdit. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_06259 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Editing Conceptual Knowledge for Large Language Models Wang, Xiaohan Mao, Shengyu Zhang, Ningyu Deng, Shumin Yao, Yunzhi Shen, Yue Liang, Lei Gu, Jinjie Chen, Huajun Computation and Language Artificial Intelligence Databases Information Retrieval Machine Learning Recently, there has been a growing interest in knowledge editing for Large Language Models (LLMs). Current approaches and evaluations merely explore the instance-level editing, while whether LLMs possess the capability to modify concepts remains unclear. This paper pioneers the investigation of editing conceptual knowledge for LLMs, by constructing a novel benchmark dataset ConceptEdit and establishing a suite of new metrics for evaluation. The experimental results reveal that, although existing editing methods can efficiently modify concept-level definition to some extent, they also have the potential to distort the related instantial knowledge in LLMs, leading to poor performance. We anticipate this can inspire further progress in better understanding LLMs. Our project homepage is available at https://zjunlp.github.io/project/ConceptEdit. |
| title | Editing Conceptual Knowledge for Large Language Models |
| topic | Computation and Language Artificial Intelligence Databases Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2403.06259 |