Analysis of Plan-based Retrieval for Grounded Text Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914917653676032 |
|---|---|
| author | Godbole, Ameya Monath, Nicholas Kim, Seungyeon Rawat, Ankit Singh McCallum, Andrew Zaheer, Manzil |
| author_facet | Godbole, Ameya Monath, Nicholas Kim, Seungyeon Rawat, Ankit Singh McCallum, Andrew Zaheer, Manzil |
| contents | In text generation, hallucinations refer to the generation of seemingly coherent text that contradicts established knowledge. One compelling hypothesis is that hallucinations occur when a language model is given a generation task outside its parametric knowledge (due to rarity, recency, domain, etc.). A common strategy to address this limitation is to infuse the language models with retrieval mechanisms, providing the model with relevant knowledge for the task. In this paper, we leverage the planning capabilities of instruction-tuned LLMs and analyze how planning can be used to guide retrieval to further reduce the frequency of hallucinations. We empirically evaluate several variations of our proposed approach on long-form text generation tasks. By improving the coverage of relevant facts, plan-guided retrieval and generation can produce more informative responses while providing a higher rate of attribution to source documents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_10490 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Analysis of Plan-based Retrieval for Grounded Text Generation Godbole, Ameya Monath, Nicholas Kim, Seungyeon Rawat, Ankit Singh McCallum, Andrew Zaheer, Manzil Computation and Language Information Retrieval In text generation, hallucinations refer to the generation of seemingly coherent text that contradicts established knowledge. One compelling hypothesis is that hallucinations occur when a language model is given a generation task outside its parametric knowledge (due to rarity, recency, domain, etc.). A common strategy to address this limitation is to infuse the language models with retrieval mechanisms, providing the model with relevant knowledge for the task. In this paper, we leverage the planning capabilities of instruction-tuned LLMs and analyze how planning can be used to guide retrieval to further reduce the frequency of hallucinations. We empirically evaluate several variations of our proposed approach on long-form text generation tasks. By improving the coverage of relevant facts, plan-guided retrieval and generation can produce more informative responses while providing a higher rate of attribution to source documents. |
| title | Analysis of Plan-based Retrieval for Grounded Text Generation |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2408.10490 |