FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917354677469184 |
|---|---|
| author | Yin, Zhihan Liang, Jianxin Wang, Yueqian Yao, Yifeng Zhang, Huishuai Zhao, Dongyan |
| author_facet | Yin, Zhihan Liang, Jianxin Wang, Yueqian Yao, Yifeng Zhang, Huishuai Zhao, Dongyan |
| contents | Multimodal Large Language Models (MLLMs) suffer from hallucinations. Existing hallucination evaluation benchmarks are often limited by over-simplified tasks leading to saturated metrics, or insufficient diversity that fails to adequately assess the hallucination extent in state-of-the-art multimodal models. To address this gap, we propose FREAK, a comprehensive multimodal benchmark designed for fine-grained hallucination assessment in MLLMs. Through high-quality photorealistic images featuring fine-grained counter-commonsense edits, FREAK innovatively evaluates hallucination phenomena in detailed visual perception of MLLMs. Extensive experiments on FREAK show severe hallucination issues in SOTA models regarding detailed visual perception. To enable deeper investigation, we curate a controlled subset to indirectly evaluate the model's ability to perceive target detailed information. Through systematic evaluation of prevailing Chain-of-Thought (CoT) prompting techniques within this task, we reveal critical insights regarding hallucination patterns and model reasoning processes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_19765 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs Yin, Zhihan Liang, Jianxin Wang, Yueqian Yao, Yifeng Zhang, Huishuai Zhao, Dongyan Computer Vision and Pattern Recognition Multimodal Large Language Models (MLLMs) suffer from hallucinations. Existing hallucination evaluation benchmarks are often limited by over-simplified tasks leading to saturated metrics, or insufficient diversity that fails to adequately assess the hallucination extent in state-of-the-art multimodal models. To address this gap, we propose FREAK, a comprehensive multimodal benchmark designed for fine-grained hallucination assessment in MLLMs. Through high-quality photorealistic images featuring fine-grained counter-commonsense edits, FREAK innovatively evaluates hallucination phenomena in detailed visual perception of MLLMs. Extensive experiments on FREAK show severe hallucination issues in SOTA models regarding detailed visual perception. To enable deeper investigation, we curate a controlled subset to indirectly evaluate the model's ability to perceive target detailed information. Through systematic evaluation of prevailing Chain-of-Thought (CoT) prompting techniques within this task, we reveal critical insights regarding hallucination patterns and model reasoning processes. |
| title | FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.19765 |