Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.01022 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929600107380736 |
|---|---|
| author | Sankararaman, Hithesh Yasin, Mohammed Nasheed Sorensen, Tanner Di Bari, Alessandro Stolcke, Andreas |
| author_facet | Sankararaman, Hithesh Yasin, Mohammed Nasheed Sorensen, Tanner Di Bari, Alessandro Stolcke, Andreas |
| contents | We present a light-weight approach for detecting nonfactual outputs from retrieval-augmented generation (RAG). Given a context and putative output, we compute a factuality score that can be thresholded to yield a binary decision to check the results of LLM-based question-answering, summarization, or other systems. Unlike factuality checkers that themselves rely on LLMs, we use compact, open-source natural language inference (NLI) models that yield a freely accessible solution with low latency and low cost at run-time, and no need for LLM fine-tuning. The approach also enables downstream mitigation and correction of hallucinations, by tracing them back to specific context chunks. Our experiments show high area under the ROC curve (AUC) across a wide range of relevant open source datasets, indicating the effectiveness of our method for fact-checking RAG output. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_01022 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output Sankararaman, Hithesh Yasin, Mohammed Nasheed Sorensen, Tanner Di Bari, Alessandro Stolcke, Andreas Computation and Language We present a light-weight approach for detecting nonfactual outputs from retrieval-augmented generation (RAG). Given a context and putative output, we compute a factuality score that can be thresholded to yield a binary decision to check the results of LLM-based question-answering, summarization, or other systems. Unlike factuality checkers that themselves rely on LLMs, we use compact, open-source natural language inference (NLI) models that yield a freely accessible solution with low latency and low cost at run-time, and no need for LLM fine-tuning. The approach also enables downstream mitigation and correction of hallucinations, by tracing them back to specific context chunks. Our experiments show high area under the ROC curve (AUC) across a wide range of relevant open source datasets, indicating the effectiveness of our method for fact-checking RAG output. |
| title | Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2411.01022 |