Latent Knowledge Scalpel: Precise and Massive Knowledge Editing for Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911093582987264 |
|---|---|
| author | Liu, Xin Song, Qiyang Xu, Shaowen Zhou, Kerou Jiang, Wenbo Jia, Xiaoqi Zhang, Weijuan Huang, Heqing Li, Yakai |
| author_facet | Liu, Xin Song, Qiyang Xu, Shaowen Zhou, Kerou Jiang, Wenbo Jia, Xiaoqi Zhang, Weijuan Huang, Heqing Li, Yakai |
| contents | Large Language Models (LLMs) often retain inaccurate or outdated information from pre-training, leading to incorrect predictions or biased outputs during inference. While existing model editing methods can address this challenge, they struggle with editing large amounts of factual information simultaneously and may compromise the general capabilities of the models. In this paper, our empirical study demonstrates that it is feasible to edit the internal representations of LLMs and replace the entities in a manner similar to editing natural language inputs. Based on this insight, we introduce the Latent Knowledge Scalpel (LKS), an LLM editor that manipulates the latent knowledge of specific entities via a lightweight hypernetwork to enable precise and large-scale editing. Experiments conducted on Llama-2 and Mistral show even with the number of simultaneous edits reaching 10,000, LKS effectively performs knowledge editing while preserving the general abilities of the edited LLMs. Code is available at: https://github.com/Linuxin-xxx/LKS. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_03741 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Latent Knowledge Scalpel: Precise and Massive Knowledge Editing for Large Language Models Liu, Xin Song, Qiyang Xu, Shaowen Zhou, Kerou Jiang, Wenbo Jia, Xiaoqi Zhang, Weijuan Huang, Heqing Li, Yakai Machine Learning Artificial Intelligence Large Language Models (LLMs) often retain inaccurate or outdated information from pre-training, leading to incorrect predictions or biased outputs during inference. While existing model editing methods can address this challenge, they struggle with editing large amounts of factual information simultaneously and may compromise the general capabilities of the models. In this paper, our empirical study demonstrates that it is feasible to edit the internal representations of LLMs and replace the entities in a manner similar to editing natural language inputs. Based on this insight, we introduce the Latent Knowledge Scalpel (LKS), an LLM editor that manipulates the latent knowledge of specific entities via a lightweight hypernetwork to enable precise and large-scale editing. Experiments conducted on Llama-2 and Mistral show even with the number of simultaneous edits reaching 10,000, LKS effectively performs knowledge editing while preserving the general abilities of the edited LLMs. Code is available at: https://github.com/Linuxin-xxx/LKS. |
| title | Latent Knowledge Scalpel: Precise and Massive Knowledge Editing for Large Language Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2508.03741 |