From Relevance to Utility: Evidence Retrieval with Feedback for Fact Verification
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908600816893952 |
|---|---|
| author | Zhang, Hengran Zhang, Ruqing Guo, Jiafeng de Rijke, Maarten Fan, Yixing Cheng, Xueqi |
| author_facet | Zhang, Hengran Zhang, Ruqing Guo, Jiafeng de Rijke, Maarten Fan, Yixing Cheng, Xueqi |
| contents | Retrieval-enhanced methods have become a primary approach in fact verification (FV); it requires reasoning over multiple retrieved pieces of evidence to verify the integrity of a claim. To retrieve evidence, existing work often employs off-the-shelf retrieval models whose design is based on the probability ranking principle. We argue that, rather than relevance, for FV we need to focus on the utility that a claim verifier derives from the retrieved evidence. We introduce the feedback-based evidence retriever(FER) that optimizes the evidence retrieval process by incorporating feedback from the claim verifier. As a feedback signal we use the divergence in utility between how effectively the verifier utilizes the retrieved evidence and the ground-truth evidence to produce the final claim label. Empirical studies demonstrate the superiority of FER over prevailing baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_11675 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | From Relevance to Utility: Evidence Retrieval with Feedback for Fact Verification Zhang, Hengran Zhang, Ruqing Guo, Jiafeng de Rijke, Maarten Fan, Yixing Cheng, Xueqi Information Retrieval Retrieval-enhanced methods have become a primary approach in fact verification (FV); it requires reasoning over multiple retrieved pieces of evidence to verify the integrity of a claim. To retrieve evidence, existing work often employs off-the-shelf retrieval models whose design is based on the probability ranking principle. We argue that, rather than relevance, for FV we need to focus on the utility that a claim verifier derives from the retrieved evidence. We introduce the feedback-based evidence retriever(FER) that optimizes the evidence retrieval process by incorporating feedback from the claim verifier. As a feedback signal we use the divergence in utility between how effectively the verifier utilizes the retrieved evidence and the ground-truth evidence to produce the final claim label. Empirical studies demonstrate the superiority of FER over prevailing baselines. |
| title | From Relevance to Utility: Evidence Retrieval with Feedback for Fact Verification |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2310.11675 |