KGValidator: A Framework for Automatic Validation of Knowledge Graph Construction
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909180413083648 |
|---|---|
| author | Boylan, Jack Mangla, Shashank Thorn, Dominic Ghalandari, Demian Gholipour Ghaffari, Parsa Hokamp, Chris |
| author_facet | Boylan, Jack Mangla, Shashank Thorn, Dominic Ghalandari, Demian Gholipour Ghaffari, Parsa Hokamp, Chris |
| contents | This study explores the use of Large Language Models (LLMs) for automatic evaluation of knowledge graph (KG) completion models. Historically, validating information in KGs has been a challenging task, requiring large-scale human annotation at prohibitive cost. With the emergence of general-purpose generative AI and LLMs, it is now plausible that human-in-the-loop validation could be replaced by a generative agent. We introduce a framework for consistency and validation when using generative models to validate knowledge graphs. Our framework is based upon recent open-source developments for structural and semantic validation of LLM outputs, and upon flexible approaches to fact checking and verification, supported by the capacity to reference external knowledge sources of any kind. The design is easy to adapt and extend, and can be used to verify any kind of graph-structured data through a combination of model-intrinsic knowledge, user-supplied context, and agents capable of external knowledge retrieval. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_15923 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | KGValidator: A Framework for Automatic Validation of Knowledge Graph Construction Boylan, Jack Mangla, Shashank Thorn, Dominic Ghalandari, Demian Gholipour Ghaffari, Parsa Hokamp, Chris Artificial Intelligence Computation and Language This study explores the use of Large Language Models (LLMs) for automatic evaluation of knowledge graph (KG) completion models. Historically, validating information in KGs has been a challenging task, requiring large-scale human annotation at prohibitive cost. With the emergence of general-purpose generative AI and LLMs, it is now plausible that human-in-the-loop validation could be replaced by a generative agent. We introduce a framework for consistency and validation when using generative models to validate knowledge graphs. Our framework is based upon recent open-source developments for structural and semantic validation of LLM outputs, and upon flexible approaches to fact checking and verification, supported by the capacity to reference external knowledge sources of any kind. The design is easy to adapt and extend, and can be used to verify any kind of graph-structured data through a combination of model-intrinsic knowledge, user-supplied context, and agents capable of external knowledge retrieval. |
| title | KGValidator: A Framework for Automatic Validation of Knowledge Graph Construction |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2404.15923 |