Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.07309 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916978282725376 |
|---|---|
| author | Huang, Yin Xu, Yifan Ethan Sun, Kai Yan, Vera Sun, Alicia Khan, Haidar Nguyen, Jimmy Chen, Jingxiang Kachuee, Mohammad Lin, Zhaojiang Liu, Yue Colak, Aaron Kumar, Anuj Yih, Wen-tau Dong, Xin Luna |
| author_facet | Huang, Yin Xu, Yifan Ethan Sun, Kai Yan, Vera Sun, Alicia Khan, Haidar Nguyen, Jimmy Chen, Jingxiang Kachuee, Mohammad Lin, Zhaojiang Liu, Yue Colak, Aaron Kumar, Anuj Yih, Wen-tau Dong, Xin Luna |
| contents | Can Large Language Models (LLMs) be trained to avoid hallucinating factual statements, and can Retrieval-Augmented Generation (RAG) be triggered only when necessary to reduce retrieval and computation costs? In this work, we address both challenges simultaneously. We introduce ConfQA, a fine-tuning strategy that reduces hallucination rates from 20-40% to below 5% across multiple factuality benchmarks. The approach is simple: when the model answers correctly, it is trained to output the answer; otherwise, it is trained to respond with "I am unsure". Two design choices make this training effective: (1) a dampening prompt ("answer only if you are confident") that explicitly discourages overconfident hallucinations, and (2) training data drawn from atomic factual statements (e.g., knowledge graph attribute values), which calibrates model confidence and yields robust generalization across domains and question types. Building on ConfQA, we propose ConfRAG, a triggering strategy that invokes RAG only when the model responses with unsure. This framework achieves accuracy above 95% in ideal case while reducing unnecessary external retrievals by over 30%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_07309 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ConfRAG: Confidence-Guided Retrieval-Augmenting Generation Huang, Yin Xu, Yifan Ethan Sun, Kai Yan, Vera Sun, Alicia Khan, Haidar Nguyen, Jimmy Chen, Jingxiang Kachuee, Mohammad Lin, Zhaojiang Liu, Yue Colak, Aaron Kumar, Anuj Yih, Wen-tau Dong, Xin Luna Computation and Language Can Large Language Models (LLMs) be trained to avoid hallucinating factual statements, and can Retrieval-Augmented Generation (RAG) be triggered only when necessary to reduce retrieval and computation costs? In this work, we address both challenges simultaneously. We introduce ConfQA, a fine-tuning strategy that reduces hallucination rates from 20-40% to below 5% across multiple factuality benchmarks. The approach is simple: when the model answers correctly, it is trained to output the answer; otherwise, it is trained to respond with "I am unsure". Two design choices make this training effective: (1) a dampening prompt ("answer only if you are confident") that explicitly discourages overconfident hallucinations, and (2) training data drawn from atomic factual statements (e.g., knowledge graph attribute values), which calibrates model confidence and yields robust generalization across domains and question types. Building on ConfQA, we propose ConfRAG, a triggering strategy that invokes RAG only when the model responses with unsure. This framework achieves accuracy above 95% in ideal case while reducing unnecessary external retrievals by over 30%. |
| title | ConfRAG: Confidence-Guided Retrieval-Augmenting Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.07309 |