SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913893271470080 |
|---|---|
| author | Chuang, Yung-Sung Cohen-Wang, Benjamin Shen, Shannon Zejiang Wu, Zhaofeng Xu, Hu Lin, Xi Victoria Glass, James Li, Shang-Wen Yih, Wen-tau |
| author_facet | Chuang, Yung-Sung Cohen-Wang, Benjamin Shen, Shannon Zejiang Wu, Zhaofeng Xu, Hu Lin, Xi Victoria Glass, James Li, Shang-Wen Yih, Wen-tau |
| contents | We introduce SelfCite, a novel self-supervised approach that aligns LLMs to generate high-quality, fine-grained, sentence-level citations for the statements in their generated responses. Instead of only relying on costly and labor-intensive annotations, SelfCite leverages a reward signal provided by the LLM itself through context ablation: If a citation is necessary, removing the cited text from the context should prevent the same response; if sufficient, retaining the cited text alone should preserve the same response. This reward can guide the inference-time best-of-N sampling strategy to improve citation quality significantly, as well as be used in preference optimization to directly fine-tune the models for generating better citations. The effectiveness of SelfCite is demonstrated by increasing citation F1 up to 5.3 points on the LongBench-Cite benchmark across five long-form question answering tasks. The source code is available at https://github.com/facebookresearch/SelfCite |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_09604 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models Chuang, Yung-Sung Cohen-Wang, Benjamin Shen, Shannon Zejiang Wu, Zhaofeng Xu, Hu Lin, Xi Victoria Glass, James Li, Shang-Wen Yih, Wen-tau Computation and Language Artificial Intelligence Machine Learning We introduce SelfCite, a novel self-supervised approach that aligns LLMs to generate high-quality, fine-grained, sentence-level citations for the statements in their generated responses. Instead of only relying on costly and labor-intensive annotations, SelfCite leverages a reward signal provided by the LLM itself through context ablation: If a citation is necessary, removing the cited text from the context should prevent the same response; if sufficient, retaining the cited text alone should preserve the same response. This reward can guide the inference-time best-of-N sampling strategy to improve citation quality significantly, as well as be used in preference optimization to directly fine-tune the models for generating better citations. The effectiveness of SelfCite is demonstrated by increasing citation F1 up to 5.3 points on the LongBench-Cite benchmark across five long-form question answering tasks. The source code is available at https://github.com/facebookresearch/SelfCite |
| title | SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2502.09604 |