Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.04495 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909016806916096 |
|---|---|
| author | Song, Zhipeng Zhou, Yizhi Kong, Xiangyu Jiao, Jiulong Ye, Xuezhou Gao, Chunqi Shi, Xueqing Zhou, Yuhang Qi, Heng |
| author_facet | Song, Zhipeng Zhou, Yizhi Kong, Xiangyu Jiao, Jiulong Ye, Xuezhou Gao, Chunqi Shi, Xueqing Zhou, Yuhang Qi, Heng |
| contents | Retrieval-Augmented Generation (RAG) depends on document ranking to provide useful evidence for generation, but conventional reranking methods mainly optimize query-document relevance rather than generation usefulness. A relevant document may still introduce noise, while a lower-ranked document may better reduce the generator's uncertainty. We propose CAR (Confidence-Aware Reranking), a query-guided, training-free, and plug-and-play reranking framework that uses generator confidence change as a document usefulness signal. CAR estimates confidence through the semantic consistency of multiple sampled answers under query-only and query-document conditions. Documents that significantly increase confidence are promoted, those that decrease confidence are demoted, and uncertain cases preserve the baseline order, while a query-level gate avoids unnecessary intervention on already confident queries. Experiments on four BEIR datasets show that CAR consistently improves NDCG@5 across sparse and dense retrievers, LLM-based and supervised rerankers, and four LLM backbones. Notably, CAR improves the YesNo reranker by 25.4 percent on average under Contriever retrieval, and its ranking gains strongly correlate with downstream generation F1 improvements, achieving Spearman rho = 0.964. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_04495 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CAR: Query-Guided Confidence-Aware Reranking for Retrieval-Augmented Generation Song, Zhipeng Zhou, Yizhi Kong, Xiangyu Jiao, Jiulong Ye, Xuezhou Gao, Chunqi Shi, Xueqing Zhou, Yuhang Qi, Heng Computation and Language Artificial Intelligence Retrieval-Augmented Generation (RAG) depends on document ranking to provide useful evidence for generation, but conventional reranking methods mainly optimize query-document relevance rather than generation usefulness. A relevant document may still introduce noise, while a lower-ranked document may better reduce the generator's uncertainty. We propose CAR (Confidence-Aware Reranking), a query-guided, training-free, and plug-and-play reranking framework that uses generator confidence change as a document usefulness signal. CAR estimates confidence through the semantic consistency of multiple sampled answers under query-only and query-document conditions. Documents that significantly increase confidence are promoted, those that decrease confidence are demoted, and uncertain cases preserve the baseline order, while a query-level gate avoids unnecessary intervention on already confident queries. Experiments on four BEIR datasets show that CAR consistently improves NDCG@5 across sparse and dense retrievers, LLM-based and supervised rerankers, and four LLM backbones. Notably, CAR improves the YesNo reranker by 25.4 percent on average under Contriever retrieval, and its ranking gains strongly correlate with downstream generation F1 improvements, achieving Spearman rho = 0.964. |
| title | CAR: Query-Guided Confidence-Aware Reranking for Retrieval-Augmented Generation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2605.04495 |