Triad: A Framework Leveraging a Multi-Role LLM-based Agent to Solve Knowledge Base Question Answering
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909328155344896 |
|---|---|
| author | Zong, Chang Yan, Yuchen Lu, Weiming Shao, Jian Huang, Eliot Chang, Heng Zhuang, Yueting |
| author_facet | Zong, Chang Yan, Yuchen Lu, Weiming Shao, Jian Huang, Eliot Chang, Heng Zhuang, Yueting |
| contents | Recent progress with LLM-based agents has shown promising results across various tasks. However, their use in answering questions from knowledge bases remains largely unexplored. Implementing a KBQA system using traditional methods is challenging due to the shortage of task-specific training data and the complexity of creating task-focused model structures. In this paper, we present Triad, a unified framework that utilizes an LLM-based agent with three roles for KBQA tasks. The agent is assigned three roles to tackle different KBQA subtasks: agent as a generalist for mastering various subtasks, as a decision maker for the selection of candidates, and as an advisor for answering questions with knowledge. Our KBQA framework is executed in four phases, involving the collaboration of the agent's multiple roles. We evaluated the performance of our framework using three benchmark datasets, and the results show that our framework outperforms state-of-the-art systems on the LC-QuAD and YAGO-QA benchmarks, yielding F1 scores of 11.8% and 20.7%, respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_14320 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Triad: A Framework Leveraging a Multi-Role LLM-based Agent to Solve Knowledge Base Question Answering Zong, Chang Yan, Yuchen Lu, Weiming Shao, Jian Huang, Eliot Chang, Heng Zhuang, Yueting Computation and Language Artificial Intelligence 68T50 I.2.7 Recent progress with LLM-based agents has shown promising results across various tasks. However, their use in answering questions from knowledge bases remains largely unexplored. Implementing a KBQA system using traditional methods is challenging due to the shortage of task-specific training data and the complexity of creating task-focused model structures. In this paper, we present Triad, a unified framework that utilizes an LLM-based agent with three roles for KBQA tasks. The agent is assigned three roles to tackle different KBQA subtasks: agent as a generalist for mastering various subtasks, as a decision maker for the selection of candidates, and as an advisor for answering questions with knowledge. Our KBQA framework is executed in four phases, involving the collaboration of the agent's multiple roles. We evaluated the performance of our framework using three benchmark datasets, and the results show that our framework outperforms state-of-the-art systems on the LC-QuAD and YAGO-QA benchmarks, yielding F1 scores of 11.8% and 20.7%, respectively. |
| title | Triad: A Framework Leveraging a Multi-Role LLM-based Agent to Solve Knowledge Base Question Answering |
| topic | Computation and Language Artificial Intelligence 68T50 I.2.7 |
| url | https://arxiv.org/abs/2402.14320 |