Semantic Similarity is a Spurious Measure of Comic Understanding: Lessons Learned from Hallucinations in a Benchmarking Experiment
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910038032908288 |
|---|---|
| author | Driggers-Ellis, Christopher Tibrewal, Nachiketh Bogulla, Rohit Khanna, Harsh Youm, Sangpil Grant, Christan Dorr, Bonnie |
| author_facet | Driggers-Ellis, Christopher Tibrewal, Nachiketh Bogulla, Rohit Khanna, Harsh Youm, Sangpil Grant, Christan Dorr, Bonnie |
| contents | A system that enables blind or visually impaired users to access comics/manga would introduce a new medium of storytelling to this community. However, no such system currently exists. Generative vision-language models (VLMs) have shown promise in describing images and understanding comics, but most research on comic understanding is limited to panel-level analysis. To fully support blind and visually impaired users, greater attention must be paid to page-level understanding and interpretation. In this work, we present a preliminary benchmark of VLM performance on comic interpretation tasks. We identify and categorize hallucinations that emerge during this process, organizing them into generalized object-hallucination taxonomies. We conclude with guidance on future research, emphasizing hallucination mitigation and improved data curation for comic interpretation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_01950 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Semantic Similarity is a Spurious Measure of Comic Understanding: Lessons Learned from Hallucinations in a Benchmarking Experiment Driggers-Ellis, Christopher Tibrewal, Nachiketh Bogulla, Rohit Khanna, Harsh Youm, Sangpil Grant, Christan Dorr, Bonnie Machine Learning Computation and Language Computer Vision and Pattern Recognition I.2.7; I.4.9 A system that enables blind or visually impaired users to access comics/manga would introduce a new medium of storytelling to this community. However, no such system currently exists. Generative vision-language models (VLMs) have shown promise in describing images and understanding comics, but most research on comic understanding is limited to panel-level analysis. To fully support blind and visually impaired users, greater attention must be paid to page-level understanding and interpretation. In this work, we present a preliminary benchmark of VLM performance on comic interpretation tasks. We identify and categorize hallucinations that emerge during this process, organizing them into generalized object-hallucination taxonomies. We conclude with guidance on future research, emphasizing hallucination mitigation and improved data curation for comic interpretation. |
| title | Semantic Similarity is a Spurious Measure of Comic Understanding: Lessons Learned from Hallucinations in a Benchmarking Experiment |
| topic | Machine Learning Computation and Language Computer Vision and Pattern Recognition I.2.7; I.4.9 |
| url | https://arxiv.org/abs/2603.01950 |