KS-LLM: Knowledge Selection of Large Language Models with Evidence Document for Question Answering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909180088025088 |
|---|---|
| author | Zheng, Xinxin Che, Feihu Wu, Jinyang Zhang, Shuai Nie, Shuai Liu, Kang Tao, Jianhua |
| author_facet | Zheng, Xinxin Che, Feihu Wu, Jinyang Zhang, Shuai Nie, Shuai Liu, Kang Tao, Jianhua |
| contents | Large language models (LLMs) suffer from the hallucination problem and face significant challenges when applied to knowledge-intensive tasks. A promising approach is to leverage evidence documents as extra supporting knowledge, which can be obtained through retrieval or generation. However, existing methods directly leverage the entire contents of the evidence document, which may introduce noise information and impair the performance of large language models. To tackle this problem, we propose a novel Knowledge Selection of Large Language Models (KS-LLM) method, aiming to identify valuable information from evidence documents. The KS-LLM approach utilizes triples to effectively select knowledge snippets from evidence documents that are beneficial to answering questions. Specifically, we first generate triples based on the input question, then select the evidence sentences most similar to triples from the evidence document, and finally combine the evidence sentences and triples to assist large language models in generating answers. Experimental comparisons on several question answering datasets, such as TriviaQA, WebQ, and NQ, demonstrate that the proposed method surpasses the baselines and achieves the best results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_15660 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | KS-LLM: Knowledge Selection of Large Language Models with Evidence Document for Question Answering Zheng, Xinxin Che, Feihu Wu, Jinyang Zhang, Shuai Nie, Shuai Liu, Kang Tao, Jianhua Computation and Language Large language models (LLMs) suffer from the hallucination problem and face significant challenges when applied to knowledge-intensive tasks. A promising approach is to leverage evidence documents as extra supporting knowledge, which can be obtained through retrieval or generation. However, existing methods directly leverage the entire contents of the evidence document, which may introduce noise information and impair the performance of large language models. To tackle this problem, we propose a novel Knowledge Selection of Large Language Models (KS-LLM) method, aiming to identify valuable information from evidence documents. The KS-LLM approach utilizes triples to effectively select knowledge snippets from evidence documents that are beneficial to answering questions. Specifically, we first generate triples based on the input question, then select the evidence sentences most similar to triples from the evidence document, and finally combine the evidence sentences and triples to assist large language models in generating answers. Experimental comparisons on several question answering datasets, such as TriviaQA, WebQ, and NQ, demonstrate that the proposed method surpasses the baselines and achieves the best results. |
| title | KS-LLM: Knowledge Selection of Large Language Models with Evidence Document for Question Answering |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2404.15660 |