GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916766765023232 |
|---|---|
| author | Luo, Kangyang Bai, Yuzhuo Gao, Cheng Si, Shuzheng Shen, Yingli Liu, Zhu Wang, Zhitong Kong, Cunliang Li, Wenhao Huang, Yufei Tian, Ye Xiong, Xuantang Han, Lei Sun, Maosong |
| author_facet | Luo, Kangyang Bai, Yuzhuo Gao, Cheng Si, Shuzheng Shen, Yingli Liu, Zhu Wang, Zhitong Kong, Cunliang Li, Wenhao Huang, Yufei Tian, Ye Xiong, Xuantang Han, Lei Sun, Maosong |
| contents | Knowledge Graph Completion (KGC), which aims to infer missing or incomplete facts, is a crucial task for KGs. However, integrating the vital structural information of KGs into Large Language Models (LLMs) and outputting predictions deterministically remains challenging. To address this, we propose a new method called GLTW, which encodes the structural information of KGs and merges it with LLMs to enhance KGC performance. Specifically, we introduce an improved Graph Transformer (iGT) that effectively encodes subgraphs with both local and global structural information and inherits the characteristics of language model, bypassing training from scratch. Also, we develop a subgraph-based multi-classification training objective, using all entities within KG as classification objects, to boost learning efficiency.Importantly, we combine iGT with an LLM that takes KG language prompts as input.Our extensive experiments on various KG datasets show that GLTW achieves significant performance gains compared to SOTA baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11471 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion Luo, Kangyang Bai, Yuzhuo Gao, Cheng Si, Shuzheng Shen, Yingli Liu, Zhu Wang, Zhitong Kong, Cunliang Li, Wenhao Huang, Yufei Tian, Ye Xiong, Xuantang Han, Lei Sun, Maosong Computation and Language Information Retrieval Knowledge Graph Completion (KGC), which aims to infer missing or incomplete facts, is a crucial task for KGs. However, integrating the vital structural information of KGs into Large Language Models (LLMs) and outputting predictions deterministically remains challenging. To address this, we propose a new method called GLTW, which encodes the structural information of KGs and merges it with LLMs to enhance KGC performance. Specifically, we introduce an improved Graph Transformer (iGT) that effectively encodes subgraphs with both local and global structural information and inherits the characteristics of language model, bypassing training from scratch. Also, we develop a subgraph-based multi-classification training objective, using all entities within KG as classification objects, to boost learning efficiency.Importantly, we combine iGT with an LLM that takes KG language prompts as input.Our extensive experiments on various KG datasets show that GLTW achieves significant performance gains compared to SOTA baselines. |
| title | GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2502.11471 |