LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917879088152576 |
|---|---|
| author | Zhu, Yuqi Wang, Xiaohan Chen, Jing Qiao, Shuofei Ou, Yixin Yao, Yunzhi Deng, Shumin Chen, Huajun Zhang, Ningyu |
| author_facet | Zhu, Yuqi Wang, Xiaohan Chen, Jing Qiao, Shuofei Ou, Yixin Yao, Yunzhi Deng, Shumin Chen, Huajun Zhang, Ningyu |
| contents | This paper presents an exhaustive quantitative and qualitative evaluation of Large Language Models (LLMs) for Knowledge Graph (KG) construction and reasoning. We engage in experiments across eight diverse datasets, focusing on four representative tasks encompassing entity and relation extraction, event extraction, link prediction, and question-answering, thereby thoroughly exploring LLMs' performance in the domain of construction and inference. Empirically, our findings suggest that LLMs, represented by GPT-4, are more suited as inference assistants rather than few-shot information extractors. Specifically, while GPT-4 exhibits good performance in tasks related to KG construction, it excels further in reasoning tasks, surpassing fine-tuned models in certain cases. Moreover, our investigation extends to the potential generalization ability of LLMs for information extraction, leading to the proposition of a Virtual Knowledge Extraction task and the development of the corresponding VINE dataset. Based on these empirical findings, we further propose AutoKG, a multi-agent-based approach employing LLMs and external sources for KG construction and reasoning. We anticipate that this research can provide invaluable insights for future undertakings in the field of knowledge graphs. The code and datasets are in https://github.com/zjunlp/AutoKG. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_13168 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities Zhu, Yuqi Wang, Xiaohan Chen, Jing Qiao, Shuofei Ou, Yixin Yao, Yunzhi Deng, Shumin Chen, Huajun Zhang, Ningyu Computation and Language Artificial Intelligence Databases Information Retrieval Machine Learning This paper presents an exhaustive quantitative and qualitative evaluation of Large Language Models (LLMs) for Knowledge Graph (KG) construction and reasoning. We engage in experiments across eight diverse datasets, focusing on four representative tasks encompassing entity and relation extraction, event extraction, link prediction, and question-answering, thereby thoroughly exploring LLMs' performance in the domain of construction and inference. Empirically, our findings suggest that LLMs, represented by GPT-4, are more suited as inference assistants rather than few-shot information extractors. Specifically, while GPT-4 exhibits good performance in tasks related to KG construction, it excels further in reasoning tasks, surpassing fine-tuned models in certain cases. Moreover, our investigation extends to the potential generalization ability of LLMs for information extraction, leading to the proposition of a Virtual Knowledge Extraction task and the development of the corresponding VINE dataset. Based on these empirical findings, we further propose AutoKG, a multi-agent-based approach employing LLMs and external sources for KG construction and reasoning. We anticipate that this research can provide invaluable insights for future undertakings in the field of knowledge graphs. The code and datasets are in https://github.com/zjunlp/AutoKG. |
| title | LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities |
| topic | Computation and Language Artificial Intelligence Databases Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2305.13168 |