CiteCheck: Towards Accurate Citation Faithfulness Detection
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910830743781376 |
|---|---|
| author | Xu, Ziyao Wei, Shaohang Han, Zhuoheng Jin, Jing Yang, Zhe Li, Xiaoguang Tan, Haochen Guo, Zhijiang Wang, Houfeng |
| author_facet | Xu, Ziyao Wei, Shaohang Han, Zhuoheng Jin, Jing Yang, Zhe Li, Xiaoguang Tan, Haochen Guo, Zhijiang Wang, Houfeng |
| contents | Citation faithfulness detection is critical for enhancing retrieval-augmented generation (RAG) systems, yet large-scale Chinese datasets for this task are scarce. Existing methods face prohibitive costs due to the need for manually annotated negative samples. To address this, we introduce the first large-scale Chinese dataset CiteCheck for citation faithfulness detection, constructed via a cost-effective approach using two-stage manual annotation. This method balances positive and negative samples while significantly reducing annotation expenses. CiteCheck comprises training and test splits. Experiments demonstrate that: (1) the test samples are highly challenging, with even state-of-the-art LLMs failing to achieve high accuracy; and (2) training data augmented with LLM-generated negative samples enables smaller models to attain strong performance using parameter-efficient fine-tuning. CiteCheck provides a robust foundation for advancing citation faithfulness detection in Chinese RAG systems. The dataset is publicly available to facilitate research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_10881 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CiteCheck: Towards Accurate Citation Faithfulness Detection Xu, Ziyao Wei, Shaohang Han, Zhuoheng Jin, Jing Yang, Zhe Li, Xiaoguang Tan, Haochen Guo, Zhijiang Wang, Houfeng Computation and Language Citation faithfulness detection is critical for enhancing retrieval-augmented generation (RAG) systems, yet large-scale Chinese datasets for this task are scarce. Existing methods face prohibitive costs due to the need for manually annotated negative samples. To address this, we introduce the first large-scale Chinese dataset CiteCheck for citation faithfulness detection, constructed via a cost-effective approach using two-stage manual annotation. This method balances positive and negative samples while significantly reducing annotation expenses. CiteCheck comprises training and test splits. Experiments demonstrate that: (1) the test samples are highly challenging, with even state-of-the-art LLMs failing to achieve high accuracy; and (2) training data augmented with LLM-generated negative samples enables smaller models to attain strong performance using parameter-efficient fine-tuning. CiteCheck provides a robust foundation for advancing citation faithfulness detection in Chinese RAG systems. The dataset is publicly available to facilitate research. |
| title | CiteCheck: Towards Accurate Citation Faithfulness Detection |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.10881 |