KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Search

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Luo, Haoran, E, Haihong, Guo, Yikai, Lin, Qika, Wu, Xiaobao, Mu, Xinyu, Liu, Wenhao, Song, Meina, Zhu, Yifan, Tuan, Luu Anh
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Databases
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866918038562930688
author	Luo, Haoran E, Haihong Guo, Yikai Lin, Qika Wu, Xiaobao Mu, Xinyu Liu, Wenhao Song, Meina Zhu, Yifan Tuan, Luu Anh
author_facet	Luo, Haoran E, Haihong Guo, Yikai Lin, Qika Wu, Xiaobao Mu, Xinyu Liu, Wenhao Song, Meina Zhu, Yifan Tuan, Luu Anh
contents	Knowledge Base Question Answering (KBQA) aims to answer natural language questions with a large-scale structured knowledge base (KB). Despite advancements with large language models (LLMs), KBQA still faces challenges in weak KB awareness, imbalance between effectiveness and efficiency, and high reliance on annotated data. To address these challenges, we propose KBQA-o1, a novel agentic KBQA method with Monte Carlo Tree Search (MCTS). It introduces a ReAct-based agent process for stepwise logical form generation with KB environment exploration. Moreover, it employs MCTS, a heuristic search method driven by policy and reward models, to balance agentic exploration's performance and search space. With heuristic exploration, KBQA-o1 generates high-quality annotations for further improvement by incremental fine-tuning. Experimental results show that KBQA-o1 outperforms previous low-resource KBQA methods with limited annotated data, boosting Llama-3.1-8B model's GrailQA F1 performance to 78.5% compared to 48.5% of the previous sota method with GPT-3.5-turbo. Our code is publicly available.
format	Preprint
id	arxiv_https___arxiv_org_abs_2501_18922
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Search Luo, Haoran E, Haihong Guo, Yikai Lin, Qika Wu, Xiaobao Mu, Xinyu Liu, Wenhao Song, Meina Zhu, Yifan Tuan, Luu Anh Computation and Language Artificial Intelligence Databases Knowledge Base Question Answering (KBQA) aims to answer natural language questions with a large-scale structured knowledge base (KB). Despite advancements with large language models (LLMs), KBQA still faces challenges in weak KB awareness, imbalance between effectiveness and efficiency, and high reliance on annotated data. To address these challenges, we propose KBQA-o1, a novel agentic KBQA method with Monte Carlo Tree Search (MCTS). It introduces a ReAct-based agent process for stepwise logical form generation with KB environment exploration. Moreover, it employs MCTS, a heuristic search method driven by policy and reward models, to balance agentic exploration's performance and search space. With heuristic exploration, KBQA-o1 generates high-quality annotations for further improvement by incremental fine-tuning. Experimental results show that KBQA-o1 outperforms previous low-resource KBQA methods with limited annotated data, boosting Llama-3.1-8B model's GrailQA F1 performance to 78.5% compared to 48.5% of the previous sota method with GPT-3.5-turbo. Our code is publicly available.
title	KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Search
topic	Computation and Language Artificial Intelligence Databases
url	https://arxiv.org/abs/2501.18922