Evaluating Knowledge Graph Based Retrieval Augmented Generation Methods under Knowledge Incompleteness
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916924318810112 |
|---|---|
| author | Zhou, Dongzhuoran Zhu, Yuqicheng Wang, Xiaxia He, Yuan Chen, Jiaoyan Staab, Steffen Kharlamov, Evgeny |
| author_facet | Zhou, Dongzhuoran Zhu, Yuqicheng Wang, Xiaxia He, Yuan Chen, Jiaoyan Staab, Steffen Kharlamov, Evgeny |
| contents | Knowledge Graph based Retrieval-Augmented Generation (KG-RAG) is a technique that enhances Large Language Model (LLM) inference in tasks like Question Answering (QA) by retrieving relevant information from knowledge graphs (KGs). However, real-world KGs are often incomplete, meaning that essential information for answering questions may be missing. Existing benchmarks do not adequately capture the impact of KG incompleteness on KG-RAG performance. In this paper, we systematically evaluate KG-RAG methods under incomplete KGs by removing triples using different methods and analyzing the resulting effects. We demonstrate that KG-RAG methods are sensitive to KG incompleteness, highlighting the need for more robust approaches in realistic settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_05163 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Knowledge Graph Based Retrieval Augmented Generation Methods under Knowledge Incompleteness Zhou, Dongzhuoran Zhu, Yuqicheng Wang, Xiaxia He, Yuan Chen, Jiaoyan Staab, Steffen Kharlamov, Evgeny Artificial Intelligence Knowledge Graph based Retrieval-Augmented Generation (KG-RAG) is a technique that enhances Large Language Model (LLM) inference in tasks like Question Answering (QA) by retrieving relevant information from knowledge graphs (KGs). However, real-world KGs are often incomplete, meaning that essential information for answering questions may be missing. Existing benchmarks do not adequately capture the impact of KG incompleteness on KG-RAG performance. In this paper, we systematically evaluate KG-RAG methods under incomplete KGs by removing triples using different methods and analyzing the resulting effects. We demonstrate that KG-RAG methods are sensitive to KG incompleteness, highlighting the need for more robust approaches in realistic settings. |
| title | Evaluating Knowledge Graph Based Retrieval Augmented Generation Methods under Knowledge Incompleteness |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2504.05163 |