Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.06956 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911047616561152 |
|---|---|
| author | Perçin, Sezen Su, Xin Syed, Qutub Sha Howard, Phillip Kuvshinov, Aleksei Schwinn, Leo Scholl, Kay-Ulrich |
| author_facet | Perçin, Sezen Su, Xin Syed, Qutub Sha Howard, Phillip Kuvshinov, Aleksei Schwinn, Leo Scholl, Kay-Ulrich |
| contents | Large language models (LLMs) are very costly and inefficient to update with new information. To address this limitation, retrieval-augmented generation (RAG) has been proposed as a solution that dynamically incorporates external knowledge during inference, improving factual consistency and reducing hallucinations. Despite its promise, RAG systems face practical challenges-most notably, a strong dependence on the quality of the input query for accurate retrieval. In this paper, we investigate the sensitivity of different components in the RAG pipeline to various types of query perturbations. Our analysis reveals that the performance of commonly used retrievers can degrade significantly even under minor query variations. We study each module in isolation as well as their combined effect in an end-to-end question answering setting, using both general-domain and domain-specific datasets. Additionally, we propose an evaluation framework to systematically assess the query-level robustness of RAG pipelines and offer actionable recommendations for practitioners based on the results of more than 1092 experiments we performed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_06956 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Investigating the Robustness of Retrieval-Augmented Generation at the Query Level Perçin, Sezen Su, Xin Syed, Qutub Sha Howard, Phillip Kuvshinov, Aleksei Schwinn, Leo Scholl, Kay-Ulrich Computation and Language Large language models (LLMs) are very costly and inefficient to update with new information. To address this limitation, retrieval-augmented generation (RAG) has been proposed as a solution that dynamically incorporates external knowledge during inference, improving factual consistency and reducing hallucinations. Despite its promise, RAG systems face practical challenges-most notably, a strong dependence on the quality of the input query for accurate retrieval. In this paper, we investigate the sensitivity of different components in the RAG pipeline to various types of query perturbations. Our analysis reveals that the performance of commonly used retrievers can degrade significantly even under minor query variations. We study each module in isolation as well as their combined effect in an end-to-end question answering setting, using both general-domain and domain-specific datasets. Additionally, we propose an evaluation framework to systematically assess the query-level robustness of RAG pipelines and offer actionable recommendations for practitioners based on the results of more than 1092 experiments we performed. |
| title | Investigating the Robustness of Retrieval-Augmented Generation at the Query Level |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.06956 |