GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913571536896000 |
|---|---|
| author | Li, Shilong He, Yancheng Guo, Hangyu Bu, Xingyuan Bai, Ge Liu, Jie Liu, Jiaheng Qu, Xingwei Li, Yangguang Ouyang, Wanli Su, Wenbo Zheng, Bo |
| author_facet | Li, Shilong He, Yancheng Guo, Hangyu Bu, Xingyuan Bai, Ge Liu, Jie Liu, Jiaheng Qu, Xingwei Li, Yangguang Ouyang, Wanli Su, Wenbo Zheng, Bo |
| contents | Long-context capabilities are essential for large language models (LLMs) to tackle complex and long-input tasks. Despite numerous efforts made to optimize LLMs for long contexts, challenges persist in robustly processing long inputs. In this paper, we introduce GraphReader, a graph-based agent system designed to handle long texts by structuring them into a graph and employing an agent to explore this graph autonomously. Upon receiving a question, the agent first undertakes a step-by-step analysis and devises a rational plan. It then invokes a set of predefined functions to read node content and neighbors, facilitating a coarse-to-fine exploration of the graph. Throughout the exploration, the agent continuously records new insights and reflects on current circumstances to optimize the process until it has gathered sufficient information to generate an answer. Experimental results on the LV-Eval dataset reveal that GraphReader, using a 4k context window, consistently outperforms GPT-4-128k across context lengths from 16k to 256k by a large margin. Additionally, our approach demonstrates superior performance on four challenging single-hop and multi-hop benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_14550 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models Li, Shilong He, Yancheng Guo, Hangyu Bu, Xingyuan Bai, Ge Liu, Jie Liu, Jiaheng Qu, Xingwei Li, Yangguang Ouyang, Wanli Su, Wenbo Zheng, Bo Computation and Language Artificial Intelligence Long-context capabilities are essential for large language models (LLMs) to tackle complex and long-input tasks. Despite numerous efforts made to optimize LLMs for long contexts, challenges persist in robustly processing long inputs. In this paper, we introduce GraphReader, a graph-based agent system designed to handle long texts by structuring them into a graph and employing an agent to explore this graph autonomously. Upon receiving a question, the agent first undertakes a step-by-step analysis and devises a rational plan. It then invokes a set of predefined functions to read node content and neighbors, facilitating a coarse-to-fine exploration of the graph. Throughout the exploration, the agent continuously records new insights and reflects on current circumstances to optimize the process until it has gathered sufficient information to generate an answer. Experimental results on the LV-Eval dataset reveal that GraphReader, using a 4k context window, consistently outperforms GPT-4-128k across context lengths from 16k to 256k by a large margin. Additionally, our approach demonstrates superior performance on four challenging single-hop and multi-hop benchmarks. |
| title | GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2406.14550 |