Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915088313614336 |
|---|---|
| author | Geng, Yuxia Chen, Jiaoyan Zeng, Yuhang Chen, Zhuo Zhang, Wen Pan, Jeff Z. Wang, Yuxiang Xu, Xiaoliang |
| author_facet | Geng, Yuxia Chen, Jiaoyan Zeng, Yuhang Chen, Zhuo Zhang, Wen Pan, Jeff Z. Wang, Yuxiang Xu, Xiaoliang |
| contents | Both graph structures and textual information play a critical role in Knowledge Graph Completion (KGC). With the success of Pre-trained Language Models (PLMs) such as BERT, they have been applied for text encoding for KGC. However, the current methods mostly prefer to fine-tune PLMs, leading to huge training costs and limited scalability to larger PLMs. In contrast, we propose to utilize prompts and perform KGC on a frozen PLM with only the prompts trained. Accordingly, we propose a new KGC method named PDKGC with two prompts -- a hard task prompt which is to adapt the KGC task to the PLM pre-training task of token prediction, and a disentangled structure prompt which learns disentangled graph representation so as to enable the PLM to combine more relevant structure knowledge with the text information. With the two prompts, PDKGC builds a textual predictor and a structural predictor, respectively, and their combination leads to more comprehensive entity prediction. Solid evaluation on three widely used KGC datasets has shown that PDKGC often outperforms the baselines including the state-of-the-art, and its components are all effective. Our codes and data are available at https://github.com/genggengcss/PDKGC. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_01837 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model Geng, Yuxia Chen, Jiaoyan Zeng, Yuhang Chen, Zhuo Zhang, Wen Pan, Jeff Z. Wang, Yuxiang Xu, Xiaoliang Computation and Language Both graph structures and textual information play a critical role in Knowledge Graph Completion (KGC). With the success of Pre-trained Language Models (PLMs) such as BERT, they have been applied for text encoding for KGC. However, the current methods mostly prefer to fine-tune PLMs, leading to huge training costs and limited scalability to larger PLMs. In contrast, we propose to utilize prompts and perform KGC on a frozen PLM with only the prompts trained. Accordingly, we propose a new KGC method named PDKGC with two prompts -- a hard task prompt which is to adapt the KGC task to the PLM pre-training task of token prediction, and a disentangled structure prompt which learns disentangled graph representation so as to enable the PLM to combine more relevant structure knowledge with the text information. With the two prompts, PDKGC builds a textual predictor and a structural predictor, respectively, and their combination leads to more comprehensive entity prediction. Solid evaluation on three widely used KGC datasets has shown that PDKGC often outperforms the baselines including the state-of-the-art, and its components are all effective. Our codes and data are available at https://github.com/genggengcss/PDKGC. |
| title | Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2312.01837 |