Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.17306 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929297652973568 |
|---|---|
| author | Rani, Anku Rawte, Vipula Sharma, Harshad Anand, Neeraj Rajbangshi, Krishnav Sheth, Amit Das, Amitava |
| author_facet | Rani, Anku Rawte, Vipula Sharma, Harshad Anand, Neeraj Rajbangshi, Krishnav Sheth, Amit Das, Amitava |
| contents | The troubling rise of hallucination presents perhaps the most significant impediment to the advancement of responsible AI. In recent times, considerable research has focused on detecting and mitigating hallucination in Large Language Models (LLMs). However, it's worth noting that hallucination is also quite prevalent in Vision-Language models (VLMs). In this paper, we offer a fine-grained discourse on profiling VLM hallucination based on two tasks: i) image captioning, and ii) Visual Question Answering (VQA). We delineate eight fine-grained orientations of visual hallucination: i) Contextual Guessing, ii) Identity Incongruity, iii) Geographical Erratum, iv) Visual Illusion, v) Gender Anomaly, vi) VLM as Classifier, vii) Wrong Reading, and viii) Numeric Discrepancy. We curate Visual HallucInation eLiciTation (VHILT), a publicly available dataset comprising 2,000 samples generated using eight VLMs across two tasks of captioning and VQA along with human annotations for the categories as mentioned earlier. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_17306 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Visual Hallucination: Definition, Quantification, and Prescriptive Remediations Rani, Anku Rawte, Vipula Sharma, Harshad Anand, Neeraj Rajbangshi, Krishnav Sheth, Amit Das, Amitava Artificial Intelligence The troubling rise of hallucination presents perhaps the most significant impediment to the advancement of responsible AI. In recent times, considerable research has focused on detecting and mitigating hallucination in Large Language Models (LLMs). However, it's worth noting that hallucination is also quite prevalent in Vision-Language models (VLMs). In this paper, we offer a fine-grained discourse on profiling VLM hallucination based on two tasks: i) image captioning, and ii) Visual Question Answering (VQA). We delineate eight fine-grained orientations of visual hallucination: i) Contextual Guessing, ii) Identity Incongruity, iii) Geographical Erratum, iv) Visual Illusion, v) Gender Anomaly, vi) VLM as Classifier, vii) Wrong Reading, and viii) Numeric Discrepancy. We curate Visual HallucInation eLiciTation (VHILT), a publicly available dataset comprising 2,000 samples generated using eight VLMs across two tasks of captioning and VQA along with human annotations for the categories as mentioned earlier. |
| title | Visual Hallucination: Definition, Quantification, and Prescriptive Remediations |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2403.17306 |