Does Knowledge Localization Hold True? Surprising Differences Between Entity and Relation Perspectives in Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910586134069248 |
|---|---|
| author | Wei, Yifan Yu, Xiaoyan Weng, Yixuan Ma, Huanhuan Zhang, Yuanzhe Zhao, Jun Liu, Kang |
| author_facet | Wei, Yifan Yu, Xiaoyan Weng, Yixuan Ma, Huanhuan Zhang, Yuanzhe Zhao, Jun Liu, Kang |
| contents | Large language models encapsulate knowledge and have demonstrated superior performance on various natural language processing tasks. Recent studies have localized this knowledge to specific model parameters, such as the MLP weights in intermediate layers. This study investigates the differences between entity and relational knowledge through knowledge editing. Our findings reveal that entity and relational knowledge cannot be directly transferred or mapped to each other. This result is unexpected, as logically, modifying the entity or the relation within the same knowledge triplet should yield equivalent outcomes. To further elucidate the differences between entity and relational knowledge, we employ causal analysis to investigate how relational knowledge is stored in pre-trained models. Contrary to prior research suggesting that knowledge is stored in MLP weights, our experiments demonstrate that relational knowledge is also significantly encoded in attention modules. This insight highlights the multifaceted nature of knowledge storage in language models, underscoring the complexity of manipulating specific types of knowledge within these models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_00617 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Does Knowledge Localization Hold True? Surprising Differences Between Entity and Relation Perspectives in Language Models Wei, Yifan Yu, Xiaoyan Weng, Yixuan Ma, Huanhuan Zhang, Yuanzhe Zhao, Jun Liu, Kang Computation and Language Artificial Intelligence Large language models encapsulate knowledge and have demonstrated superior performance on various natural language processing tasks. Recent studies have localized this knowledge to specific model parameters, such as the MLP weights in intermediate layers. This study investigates the differences between entity and relational knowledge through knowledge editing. Our findings reveal that entity and relational knowledge cannot be directly transferred or mapped to each other. This result is unexpected, as logically, modifying the entity or the relation within the same knowledge triplet should yield equivalent outcomes. To further elucidate the differences between entity and relational knowledge, we employ causal analysis to investigate how relational knowledge is stored in pre-trained models. Contrary to prior research suggesting that knowledge is stored in MLP weights, our experiments demonstrate that relational knowledge is also significantly encoded in attention modules. This insight highlights the multifaceted nature of knowledge storage in language models, underscoring the complexity of manipulating specific types of knowledge within these models. |
| title | Does Knowledge Localization Hold True? Surprising Differences Between Entity and Relation Perspectives in Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.00617 |