TARAC: Mitigating Hallucination in LVLMs via Temporal Attention Real-time Accumulative Connection
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915934130667520 |
|---|---|
| author | Jiang, Lei Xie, Chunzhao Liu, Tongxuan Zeng, Yuting Guo, jinrong Shen, Yunheng Huang, Weizhe Li, Jing Xu, Xiaohua |
| author_facet | Jiang, Lei Xie, Chunzhao Liu, Tongxuan Zeng, Yuting Guo, jinrong Shen, Yunheng Huang, Weizhe Li, Jing Xu, Xiaohua |
| contents | Large Vision-Language Models have demonstrated remarkable capabilities, yet they suffer from hallucinations that limit practical deployment. While various mitigation strategies exist, they often incur high computational overhead or require extensive retraining. In this paper, we address the issue of visual attention decay during generation, a key factor contributing to hallucinations. We propose Temporal Attention Real-time Accumulative Connection (TARAC), a novel training-free framework that dynamically accumulates and re-injects historical attention to sustain visual grounding. Inspired by cognitive reinforcement mechanisms, TARAC operates as a lightweight, plug-and-play module. Extensive experiments across diverse models (e.g., LLaVA, Qwen2-VL) and benchmarks demonstrate that TARAC significantly outperforms state-of-the-art methods. Remarkably, it achieves these gains with negligible inference overhead ($\sim$4\% TPOT increase), compared to the substantial costs of existing training-free baselines. Specifically, TARAC reduces hallucinated sentences by 25.2\% on CHAIR and improves Perception score by +10.65 on MME, validating its effectiveness and efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_04099 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TARAC: Mitigating Hallucination in LVLMs via Temporal Attention Real-time Accumulative Connection Jiang, Lei Xie, Chunzhao Liu, Tongxuan Zeng, Yuting Guo, jinrong Shen, Yunheng Huang, Weizhe Li, Jing Xu, Xiaohua Computer Vision and Pattern Recognition Artificial Intelligence Large Vision-Language Models have demonstrated remarkable capabilities, yet they suffer from hallucinations that limit practical deployment. While various mitigation strategies exist, they often incur high computational overhead or require extensive retraining. In this paper, we address the issue of visual attention decay during generation, a key factor contributing to hallucinations. We propose Temporal Attention Real-time Accumulative Connection (TARAC), a novel training-free framework that dynamically accumulates and re-injects historical attention to sustain visual grounding. Inspired by cognitive reinforcement mechanisms, TARAC operates as a lightweight, plug-and-play module. Extensive experiments across diverse models (e.g., LLaVA, Qwen2-VL) and benchmarks demonstrate that TARAC significantly outperforms state-of-the-art methods. Remarkably, it achieves these gains with negligible inference overhead ($\sim$4\% TPOT increase), compared to the substantial costs of existing training-free baselines. Specifically, TARAC reduces hallucinated sentences by 25.2\% on CHAIR and improves Perception score by +10.65 on MME, validating its effectiveness and efficiency. |
| title | TARAC: Mitigating Hallucination in LVLMs via Temporal Attention Real-time Accumulative Connection |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2504.04099 |