XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915289373868032 |
|---|---|
| author | Mao, Qianren Luo, Yangyifei Zhang, Qili Luo, Yashuo Cao, Zhilong Zhang, Jinlong Hao, HanWen Chen, Zhijun Jiang, Weifeng Liu, Junnan Wang, Xiaolong Huang, Zhenting Tan, Zhixing Jie, Sun Li, Bo Liu, Xudong Zhang, Richong Li, Jianxin |
| author_facet | Mao, Qianren Luo, Yangyifei Zhang, Qili Luo, Yashuo Cao, Zhilong Zhang, Jinlong Hao, HanWen Chen, Zhijun Jiang, Weifeng Liu, Junnan Wang, Xiaolong Huang, Zhenting Tan, Zhixing Jie, Sun Li, Bo Liu, Xudong Zhang, Richong Li, Jianxin |
| contents | Retrieval-augmented generation (RAG) synergizes the retrieval of pertinent data with the generative capabilities of Large Language Models (LLMs), ensuring that the generated output is not only contextually relevant but also accurate and current. We introduce XRAG, an open-source, modular codebase that facilitates exhaustive evaluation of the performance of foundational components of advanced RAG modules. These components are systematically categorized into four core phases: pre-retrieval, retrieval, post-retrieval, and generation. We systematically analyse them across reconfigured datasets, providing a comprehensive benchmark for their effectiveness. As the complexity of RAG systems continues to escalate, we underscore the critical need to identify potential failure points in RAG systems. We formulate a suite of experimental methodologies and diagnostic testing protocols to dissect the failure points inherent in RAG engineering. Subsequently, we proffer bespoke solutions aimed at bolstering the overall performance of these modules. Our work thoroughly evaluates the performance of advanced core components in RAG systems, providing insights into optimizations for prevalent failure points. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15529 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation Mao, Qianren Luo, Yangyifei Zhang, Qili Luo, Yashuo Cao, Zhilong Zhang, Jinlong Hao, HanWen Chen, Zhijun Jiang, Weifeng Liu, Junnan Wang, Xiaolong Huang, Zhenting Tan, Zhixing Jie, Sun Li, Bo Liu, Xudong Zhang, Richong Li, Jianxin Computation and Language Artificial Intelligence Retrieval-augmented generation (RAG) synergizes the retrieval of pertinent data with the generative capabilities of Large Language Models (LLMs), ensuring that the generated output is not only contextually relevant but also accurate and current. We introduce XRAG, an open-source, modular codebase that facilitates exhaustive evaluation of the performance of foundational components of advanced RAG modules. These components are systematically categorized into four core phases: pre-retrieval, retrieval, post-retrieval, and generation. We systematically analyse them across reconfigured datasets, providing a comprehensive benchmark for their effectiveness. As the complexity of RAG systems continues to escalate, we underscore the critical need to identify potential failure points in RAG systems. We formulate a suite of experimental methodologies and diagnostic testing protocols to dissect the failure points inherent in RAG engineering. Subsequently, we proffer bespoke solutions aimed at bolstering the overall performance of these modules. Our work thoroughly evaluates the performance of advanced core components in RAG systems, providing insights into optimizations for prevalent failure points. |
| title | XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2412.15529 |