Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2312.14346 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917629248143360 |
|---|---|
| author | Vakharia, Priyesh Joshi, Devavrat Chavan, Meenal Sonawane, Dhananjay Garg, Bhrigu Mazaheri, Parsa |
| author_facet | Vakharia, Priyesh Joshi, Devavrat Chavan, Meenal Sonawane, Dhananjay Garg, Bhrigu Mazaheri, Parsa |
| contents | Large Language Models (LLMs) are adept at text manipulation -- tasks such as machine translation and text summarization. However, these models can also be prone to hallucination, which can be detrimental to the faithfulness of any answers that the model provides. Recent works in combating hallucinations in LLMs deal with identifying hallucinated sentences and categorizing the different ways in which models hallucinate. This paper takes a deep dive into LLM behavior with respect to hallucinations, defines a token-level approach to identifying different kinds of hallucinations, and further utilizes this token-level tagging to improve the interpretability and faithfulness of LLMs in dialogue summarization tasks. Through this, the paper presents a new, enhanced dataset and a new training paradigm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_14346 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Don't Believe Everything You Read: Enhancing Summarization Interpretability through Automatic Identification of Hallucinations in Large Language Models Vakharia, Priyesh Joshi, Devavrat Chavan, Meenal Sonawane, Dhananjay Garg, Bhrigu Mazaheri, Parsa Computation and Language Artificial Intelligence Large Language Models (LLMs) are adept at text manipulation -- tasks such as machine translation and text summarization. However, these models can also be prone to hallucination, which can be detrimental to the faithfulness of any answers that the model provides. Recent works in combating hallucinations in LLMs deal with identifying hallucinated sentences and categorizing the different ways in which models hallucinate. This paper takes a deep dive into LLM behavior with respect to hallucinations, defines a token-level approach to identifying different kinds of hallucinations, and further utilizes this token-level tagging to improve the interpretability and faithfulness of LLMs in dialogue summarization tasks. Through this, the paper presents a new, enhanced dataset and a new training paradigm. |
| title | Don't Believe Everything You Read: Enhancing Summarization Interpretability through Automatic Identification of Hallucinations in Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2312.14346 |