Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908398008664064 |
|---|---|
| author | Chae, Kyubyung Choi, Jaepill Jo, Yohan Kim, Taesup |
| author_facet | Chae, Kyubyung Choi, Jaepill Jo, Yohan Kim, Taesup |
| contents | A primary challenge in abstractive summarization is hallucination -- the phenomenon where a model generates plausible text that is absent in the source text. We hypothesize that the domain (or topic) of the source text triggers the model to generate text that is highly probable in the domain, neglecting the details of the source text. To alleviate this model bias, we introduce a decoding strategy based on domain-conditional pointwise mutual information. This strategy adjusts the generation probability of each token by comparing it with the token's marginal probability within the domain of the source text. According to evaluation on the XSUM dataset, our method demonstrates improvement in terms of faithfulness and source relevance. The code is publicly available at \url{https://github.com/qqplot/dcpmi}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_09480 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information Chae, Kyubyung Choi, Jaepill Jo, Yohan Kim, Taesup Computation and Language Artificial Intelligence A primary challenge in abstractive summarization is hallucination -- the phenomenon where a model generates plausible text that is absent in the source text. We hypothesize that the domain (or topic) of the source text triggers the model to generate text that is highly probable in the domain, neglecting the details of the source text. To alleviate this model bias, we introduce a decoding strategy based on domain-conditional pointwise mutual information. This strategy adjusts the generation probability of each token by comparing it with the token's marginal probability within the domain of the source text. According to evaluation on the XSUM dataset, our method demonstrates improvement in terms of faithfulness and source relevance. The code is publicly available at \url{https://github.com/qqplot/dcpmi}. |
| title | Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2404.09480 |