Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.17481 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912598981607424 |
|---|---|
| author | Wang, Xingqi Cui, Yiming Yao, Xin Wang, Shijin Hu, Guoping Qin, Xiaoyu |
| author_facet | Wang, Xingqi Cui, Yiming Yao, Xin Wang, Shijin Hu, Guoping Qin, Xiaoyu |
| contents | Large Vision-Language Models (LVLMs) have recently demonstrated remarkable progress, yet hallucination remains a critical barrier, particularly in chart understanding, which requires sophisticated perceptual and cognitive abilities as well as rigorous factual accuracy. While prior work has investigated hallucinations and chart comprehension independently, their intersection remains largely unexplored. To address this gap, we present ChartHal, a benchmark that features a fine-grained taxonomy of hallucination scenarios in chart understanding, along with a human-validated dataset of 1,062 samples. Our evaluation shows that state-of-the-art LVLMs suffer from severe hallucinations on ChartHal, including proprietary models such as GPT-5 and o4-mini, which achieve only 34.46% and 22.79% accuracy, respectively. Further analysis reveals that questions involving information absent from or contradictory to charts are especially likely to trigger hallucinations, underscoring the urgent need for more robust mitigation strategies. Code and data are available at https://github.com/ymcui/ChartHal . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_17481 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ChartHal: A Fine-grained Framework Evaluating Hallucination of Large Vision Language Models in Chart Understanding Wang, Xingqi Cui, Yiming Yao, Xin Wang, Shijin Hu, Guoping Qin, Xiaoyu Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Large Vision-Language Models (LVLMs) have recently demonstrated remarkable progress, yet hallucination remains a critical barrier, particularly in chart understanding, which requires sophisticated perceptual and cognitive abilities as well as rigorous factual accuracy. While prior work has investigated hallucinations and chart comprehension independently, their intersection remains largely unexplored. To address this gap, we present ChartHal, a benchmark that features a fine-grained taxonomy of hallucination scenarios in chart understanding, along with a human-validated dataset of 1,062 samples. Our evaluation shows that state-of-the-art LVLMs suffer from severe hallucinations on ChartHal, including proprietary models such as GPT-5 and o4-mini, which achieve only 34.46% and 22.79% accuracy, respectively. Further analysis reveals that questions involving information absent from or contradictory to charts are especially likely to trigger hallucinations, underscoring the urgent need for more robust mitigation strategies. Code and data are available at https://github.com/ymcui/ChartHal . |
| title | ChartHal: A Fine-grained Framework Evaluating Hallucination of Large Vision Language Models in Chart Understanding |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2509.17481 |