Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917538363867136 |
|---|---|
| author | Qian, Pin Wang, Su Wang, Xiaoyuan Chen, Yihang Xu, Wenxuan Yu, Qiaolin Lin, Shuhuai Zhang, Sipeng You, Junxian Wei, Xinpeng |
| author_facet | Qian, Pin Wang, Su Wang, Xiaoyuan Chen, Yihang Xu, Wenxuan Yu, Qiaolin Lin, Shuhuai Zhang, Sipeng You, Junxian Wei, Xinpeng |
| contents | Cited RAG evaluation often treats visible sources as a grounding signal, but a real, topically relevant citation can still under-warrant the attached wording. We study this diagnostic failure as citation laundering: a related source is presented as warrant for an over-strong claim. We introduce FORCEBENCH, a contrastive stress test for evidence-force calibration. Each item holds a cited passage fixed and pairs an evidence-calibrated claim with a localized force-raised variant across five operational axes: relation, modality, scope, temporal validity, and numeric specificity. A calibrated evaluator should score the evidence-calibrated claim higher. Headline experiments use a fixed, locality-filtered 198-pair evaluation set. A citation-presence sanity check is uninformative by design; token and entity overlap still violate monotonicity on 32.8--36.4% of pairs. Across four reported model judges, standard generic support prompting is insufficient for this force-calibration stress test (aggregate MVR 47.2%), while explicit warrant-strength prompting lowers MVR to 24.5% but remains imperfect. We release the benchmark, prompts, outputs, and plug-in pipeline so citation evaluators can report monotonicity violation rate and force sensitivity alongside conventional support metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_28044 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG Qian, Pin Wang, Su Wang, Xiaoyuan Chen, Yihang Xu, Wenxuan Yu, Qiaolin Lin, Shuhuai Zhang, Sipeng You, Junxian Wei, Xinpeng Artificial Intelligence Cited RAG evaluation often treats visible sources as a grounding signal, but a real, topically relevant citation can still under-warrant the attached wording. We study this diagnostic failure as citation laundering: a related source is presented as warrant for an over-strong claim. We introduce FORCEBENCH, a contrastive stress test for evidence-force calibration. Each item holds a cited passage fixed and pairs an evidence-calibrated claim with a localized force-raised variant across five operational axes: relation, modality, scope, temporal validity, and numeric specificity. A calibrated evaluator should score the evidence-calibrated claim higher. Headline experiments use a fixed, locality-filtered 198-pair evaluation set. A citation-presence sanity check is uninformative by design; token and entity overlap still violate monotonicity on 32.8--36.4% of pairs. Across four reported model judges, standard generic support prompting is insufficient for this force-calibration stress test (aggregate MVR 47.2%), while explicit warrant-strength prompting lowers MVR to 24.5% but remains imperfect. We release the benchmark, prompts, outputs, and plug-in pipeline so citation evaluators can report monotonicity violation rate and force sensitivity alongside conventional support metrics. |
| title | Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2605.28044 |