Advancing Large Language Model Attribution through Self-Improving
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929547685920768 |
|---|---|
| author | Huang, Lei Feng, Xiaocheng Ma, Weitao Zhao, Liang Fan, Yuchun Zhong, Weihong Xu, Dongliang Yang, Qing Liu, Hongtao Qin, Bing |
| author_facet | Huang, Lei Feng, Xiaocheng Ma, Weitao Zhao, Liang Fan, Yuchun Zhong, Weihong Xu, Dongliang Yang, Qing Liu, Hongtao Qin, Bing |
| contents | Teaching large language models (LLMs) to generate text with citations to evidence sources can mitigate hallucinations and enhance verifiability in information-seeking systems. However, improving this capability requires high-quality attribution data, which is costly and labor-intensive. Inspired by recent advances in self-improvement that enhance LLMs without manual annotation, we present START, a Self-Taught AttRibuTion framework for iteratively improving the attribution capability of LLMs. First, to prevent models from stagnating due to initially insufficient supervision signals, START leverages the model to self-construct synthetic training data for warming up. To further self-improve the model's attribution ability, START iteratively utilizes fine-grained preference supervision signals constructed from its sampled responses to encourage robust, comprehensive, and attributable generation. Experiments on three open-domain question-answering datasets, covering long-form QA and multi-step reasoning, demonstrate significant performance gains of 25.13% on average without relying on human annotations and more advanced models. Further analysis reveals that START excels in aggregating information across multiple sources. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_13298 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Advancing Large Language Model Attribution through Self-Improving Huang, Lei Feng, Xiaocheng Ma, Weitao Zhao, Liang Fan, Yuchun Zhong, Weihong Xu, Dongliang Yang, Qing Liu, Hongtao Qin, Bing Computation and Language Artificial Intelligence Teaching large language models (LLMs) to generate text with citations to evidence sources can mitigate hallucinations and enhance verifiability in information-seeking systems. However, improving this capability requires high-quality attribution data, which is costly and labor-intensive. Inspired by recent advances in self-improvement that enhance LLMs without manual annotation, we present START, a Self-Taught AttRibuTion framework for iteratively improving the attribution capability of LLMs. First, to prevent models from stagnating due to initially insufficient supervision signals, START leverages the model to self-construct synthetic training data for warming up. To further self-improve the model's attribution ability, START iteratively utilizes fine-grained preference supervision signals constructed from its sampled responses to encourage robust, comprehensive, and attributable generation. Experiments on three open-domain question-answering datasets, covering long-form QA and multi-step reasoning, demonstrate significant performance gains of 25.13% on average without relying on human annotations and more advanced models. Further analysis reveals that START excels in aggregating information across multiple sources. |
| title | Advancing Large Language Model Attribution through Self-Improving |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2410.13298 |