AHA: Aligning Large Audio-Language Models for Reasoning Hallucinations via Counterfactual Hard Negatives
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908746379165696 |
|---|---|
| author | Chen, Yanxi Zhu, Wenhui Chen, Xiwen Wang, Zhipeng Li, Xin Qiu, Peijie Wang, Hao Dong, Xuanzhao Xiong, Yujian Schneider, Anderson Nevmyvaka, Yuriy Wang, Yalin |
| author_facet | Chen, Yanxi Zhu, Wenhui Chen, Xiwen Wang, Zhipeng Li, Xin Qiu, Peijie Wang, Hao Dong, Xuanzhao Xiong, Yujian Schneider, Anderson Nevmyvaka, Yuriy Wang, Yalin |
| contents | Although Large Audio-Language Models (LALMs) deliver state-of-the-art (SOTA) performance, they frequently suffer from hallucinations, e.g. generating text not grounded in the audio input. We analyze these grounding failures and identify a distinct taxonomy: Event Omission, False Event Identity, Temporal Relation Error, and Quantitative Temporal Error. To address this, we introduce the AHA (Audio Hallucination Alignment) framework. By leveraging counterfactual hard negative mining, our pipeline constructs a high-quality preference dataset that forces models to distinguish strict acoustic evidence from linguistically plausible fabrications. Additionally, we establish AHA-Eval, a diagnostic benchmark designed to rigorously test these fine-grained temporal reasoning capabilities. We apply this data to align Qwen2.5-Omni. The resulting model, Qwen-Audio-AHA, achieves a 13.7% improvement on AHA-Eval. Crucially, this benefit generalizes beyond our diagnostic set. Our model shows substantial gains on public benchmarks, including 1.3% on MMAU-Test and 1.6% on MMAR, outperforming latest SOTA methods. The model and dataset are open-sourced at https://github.com/LLM-VLM-GSL/AHA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_24052 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AHA: Aligning Large Audio-Language Models for Reasoning Hallucinations via Counterfactual Hard Negatives Chen, Yanxi Zhu, Wenhui Chen, Xiwen Wang, Zhipeng Li, Xin Qiu, Peijie Wang, Hao Dong, Xuanzhao Xiong, Yujian Schneider, Anderson Nevmyvaka, Yuriy Wang, Yalin Sound Artificial Intelligence Computation and Language Multimedia Although Large Audio-Language Models (LALMs) deliver state-of-the-art (SOTA) performance, they frequently suffer from hallucinations, e.g. generating text not grounded in the audio input. We analyze these grounding failures and identify a distinct taxonomy: Event Omission, False Event Identity, Temporal Relation Error, and Quantitative Temporal Error. To address this, we introduce the AHA (Audio Hallucination Alignment) framework. By leveraging counterfactual hard negative mining, our pipeline constructs a high-quality preference dataset that forces models to distinguish strict acoustic evidence from linguistically plausible fabrications. Additionally, we establish AHA-Eval, a diagnostic benchmark designed to rigorously test these fine-grained temporal reasoning capabilities. We apply this data to align Qwen2.5-Omni. The resulting model, Qwen-Audio-AHA, achieves a 13.7% improvement on AHA-Eval. Crucially, this benefit generalizes beyond our diagnostic set. Our model shows substantial gains on public benchmarks, including 1.3% on MMAU-Test and 1.6% on MMAR, outperforming latest SOTA methods. The model and dataset are open-sourced at https://github.com/LLM-VLM-GSL/AHA. |
| title | AHA: Aligning Large Audio-Language Models for Reasoning Hallucinations via Counterfactual Hard Negatives |
| topic | Sound Artificial Intelligence Computation and Language Multimedia |
| url | https://arxiv.org/abs/2512.24052 |