Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.08963 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912467526877184 |
|---|---|
| author | Wang, Yu Das, Kamalika Gao, Xiang Cui, Wendi Li, Peng Zhang, Jiaxin |
| author_facet | Wang, Yu Das, Kamalika Gao, Xiang Cui, Wendi Li, Peng Zhang, Jiaxin |
| contents | In tasks like summarization and open-book question answering (QA), Large Language Models (LLMs) often encounter "contextual hallucination", where they produce irrelevant or incorrect responses despite having access to accurate source information. This typically occurs because these models tend to prioritize self-generated content over the input context, causing them to disregard pertinent details. To address this challenge, we introduce a novel method called "Guided Attention Map Editing" (GAME), which dynamically adjusts attention maps to improve contextual relevance. During inference, GAME employs a trained classifier to identify attention maps prone to inducing hallucinations and executes targeted interventions. These interventions, guided by gradient-informed "edit directions'', strategically redistribute attention weights across various heads to effectively reduce hallucination. Comprehensive evaluations on challenging summarization and open-book QA tasks show that GAME consistently reduces hallucinations across a variety of open-source models. Specifically, GAME reduces hallucinations by 10% in the XSum summarization task while achieving a 7X speed-up in computational efficiency compared to the state-of-the-art baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_08963 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Gradient-guided Attention Map Editing: Towards Efficient Contextual Hallucination Mitigation Wang, Yu Das, Kamalika Gao, Xiang Cui, Wendi Li, Peng Zhang, Jiaxin Computation and Language Machine Learning In tasks like summarization and open-book question answering (QA), Large Language Models (LLMs) often encounter "contextual hallucination", where they produce irrelevant or incorrect responses despite having access to accurate source information. This typically occurs because these models tend to prioritize self-generated content over the input context, causing them to disregard pertinent details. To address this challenge, we introduce a novel method called "Guided Attention Map Editing" (GAME), which dynamically adjusts attention maps to improve contextual relevance. During inference, GAME employs a trained classifier to identify attention maps prone to inducing hallucinations and executes targeted interventions. These interventions, guided by gradient-informed "edit directions'', strategically redistribute attention weights across various heads to effectively reduce hallucination. Comprehensive evaluations on challenging summarization and open-book QA tasks show that GAME consistently reduces hallucinations across a variety of open-source models. Specifically, GAME reduces hallucinations by 10% in the XSum summarization task while achieving a 7X speed-up in computational efficiency compared to the state-of-the-art baselines. |
| title | Gradient-guided Attention Map Editing: Towards Efficient Contextual Hallucination Mitigation |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2503.08963 |