FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914850859384832 |
|---|---|
| author | Singh, Shubhankar Chaurasia, Purvi Varun, Yerram Pandya, Pranshu Gupta, Vatsal Gupta, Vivek Roth, Dan |
| author_facet | Singh, Shubhankar Chaurasia, Purvi Varun, Yerram Pandya, Pranshu Gupta, Vatsal Gupta, Vivek Roth, Dan |
| contents | Existing benchmarks for visual question answering lack in visual grounding and complexity, particularly in evaluating spatial reasoning skills. We introduce FlowVQA, a novel benchmark aimed at assessing the capabilities of visual question-answering multimodal language models in reasoning with flowcharts as visual contexts. FlowVQA comprises 2,272 carefully generated and human-verified flowchart images from three distinct content sources, along with 22,413 diverse question-answer pairs, to test a spectrum of reasoning tasks, including information localization, decision-making, and logical progression. We conduct a thorough baseline evaluation on a suite of both open-source and proprietary multimodal language models using various strategies, followed by an analysis of directional bias. The results underscore the benchmark's potential as a vital tool for advancing the field of multimodal modeling, providing a focused and challenging environment for enhancing model performance in visual and logical reasoning tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_19237 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts Singh, Shubhankar Chaurasia, Purvi Varun, Yerram Pandya, Pranshu Gupta, Vatsal Gupta, Vivek Roth, Dan Computation and Language Computer Vision and Pattern Recognition Information Retrieval Machine Learning Existing benchmarks for visual question answering lack in visual grounding and complexity, particularly in evaluating spatial reasoning skills. We introduce FlowVQA, a novel benchmark aimed at assessing the capabilities of visual question-answering multimodal language models in reasoning with flowcharts as visual contexts. FlowVQA comprises 2,272 carefully generated and human-verified flowchart images from three distinct content sources, along with 22,413 diverse question-answer pairs, to test a spectrum of reasoning tasks, including information localization, decision-making, and logical progression. We conduct a thorough baseline evaluation on a suite of both open-source and proprietary multimodal language models using various strategies, followed by an analysis of directional bias. The results underscore the benchmark's potential as a vital tool for advancing the field of multimodal modeling, providing a focused and challenging environment for enhancing model performance in visual and logical reasoning tasks. |
| title | FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts |
| topic | Computation and Language Computer Vision and Pattern Recognition Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2406.19237 |