Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.00437 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915276731187200 |
|---|---|
| author | Jiang, Yun Xie, Zilong Zhang, Wei Fang, Yun Pan, Shuai |
| author_facet | Jiang, Yun Xie, Zilong Zhang, Wei Fang, Yun Pan, Shuai |
| contents | Retrieval-augmented generation methods often neglect the quality of content retrieved from external knowledge bases, resulting in irrelevant information or potential misinformation that negatively affects the generation results of large language models. In this paper, we propose an end-to-end model with adaptive filtering for retrieval-augmented generation (E2E-AFG), which integrates answer existence judgment and text generation into a single end-to-end framework. This enables the model to focus more effectively on relevant content while reducing the influence of irrelevant information and generating accurate answers. We evaluate E2E-AFG on six representative knowledge-intensive language datasets, and the results show that it consistently outperforms baseline models across all tasks, demonstrating the effectiveness and robustness of the proposed approach. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_00437 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | E2E-AFG: An End-to-End Model with Adaptive Filtering for Retrieval-Augmented Generation Jiang, Yun Xie, Zilong Zhang, Wei Fang, Yun Pan, Shuai Computation and Language Artificial Intelligence Retrieval-augmented generation methods often neglect the quality of content retrieved from external knowledge bases, resulting in irrelevant information or potential misinformation that negatively affects the generation results of large language models. In this paper, we propose an end-to-end model with adaptive filtering for retrieval-augmented generation (E2E-AFG), which integrates answer existence judgment and text generation into a single end-to-end framework. This enables the model to focus more effectively on relevant content while reducing the influence of irrelevant information and generating accurate answers. We evaluate E2E-AFG on six representative knowledge-intensive language datasets, and the results show that it consistently outperforms baseline models across all tasks, demonstrating the effectiveness and robustness of the proposed approach. |
| title | E2E-AFG: An End-to-End Model with Adaptive Filtering for Retrieval-Augmented Generation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2411.00437 |