CiteEval: Principle-Driven Citation Evaluation for Source Attribution
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912409574178816 |
|---|---|
| author | Xu, Yumo Qi, Peng Chen, Jifan Liu, Kunlun Han, Rujun Liu, Lan Min, Bonan Castelli, Vittorio Gupta, Arshit Wang, Zhiguo |
| author_facet | Xu, Yumo Qi, Peng Chen, Jifan Liu, Kunlun Han, Rujun Liu, Lan Min, Bonan Castelli, Vittorio Gupta, Arshit Wang, Zhiguo |
| contents | Citation quality is crucial in information-seeking systems, directly influencing trust and the effectiveness of information access. Current evaluation frameworks, both human and automatic, mainly rely on Natural Language Inference (NLI) to assess binary or ternary supportiveness from cited sources, which we argue is a suboptimal proxy for citation evaluation. In this work we introduce CiteEval, a citation evaluation framework driven by principles focusing on fine-grained citation assessment within a broad context, encompassing not only the cited sources but the full retrieval context, user query, and generated text. Guided by the proposed framework, we construct CiteBench, a multi-domain benchmark with high-quality human annotations on citation quality. To enable efficient evaluation, we further develop CiteEval-Auto, a suite of model-based metrics that exhibit strong correlation with human judgments. Experiments across diverse systems demonstrate CiteEval-Auto's superior ability to capture the multifaceted nature of citations compared to existing metrics, offering a principled and scalable approach to evaluate and improve model-generated citations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_01829 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CiteEval: Principle-Driven Citation Evaluation for Source Attribution Xu, Yumo Qi, Peng Chen, Jifan Liu, Kunlun Han, Rujun Liu, Lan Min, Bonan Castelli, Vittorio Gupta, Arshit Wang, Zhiguo Computation and Language Artificial Intelligence Information Retrieval Citation quality is crucial in information-seeking systems, directly influencing trust and the effectiveness of information access. Current evaluation frameworks, both human and automatic, mainly rely on Natural Language Inference (NLI) to assess binary or ternary supportiveness from cited sources, which we argue is a suboptimal proxy for citation evaluation. In this work we introduce CiteEval, a citation evaluation framework driven by principles focusing on fine-grained citation assessment within a broad context, encompassing not only the cited sources but the full retrieval context, user query, and generated text. Guided by the proposed framework, we construct CiteBench, a multi-domain benchmark with high-quality human annotations on citation quality. To enable efficient evaluation, we further develop CiteEval-Auto, a suite of model-based metrics that exhibit strong correlation with human judgments. Experiments across diverse systems demonstrate CiteEval-Auto's superior ability to capture the multifaceted nature of citations compared to existing metrics, offering a principled and scalable approach to evaluate and improve model-generated citations. |
| title | CiteEval: Principle-Driven Citation Evaluation for Source Attribution |
| topic | Computation and Language Artificial Intelligence Information Retrieval |
| url | https://arxiv.org/abs/2506.01829 |