DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909751076454400 |
|---|---|
| author | Kartha, Aaryaman Masry, Ahmed Islam, Mohammed Saidul Lang, Thinh Rahman, Shadikur Mahbub, Ridwan Rahman, Mizanur Ahmed, Mahir Parvez, Md Rizwan Hoque, Enamul Joty, Shafiq |
| author_facet | Kartha, Aaryaman Masry, Ahmed Islam, Mohammed Saidul Lang, Thinh Rahman, Shadikur Mahbub, Ridwan Rahman, Mizanur Ahmed, Mahir Parvez, Md Rizwan Hoque, Enamul Joty, Shafiq |
| contents | Dashboards are powerful visualization tools for data-driven decision-making, integrating multiple interactive views that allow users to explore, filter, and navigate data. Unlike static charts, dashboards support rich interactivity, which is essential for uncovering insights in real-world analytical workflows. However, existing question-answering benchmarks for data visualizations largely overlook this interactivity, focusing instead on static charts. This limitation severely constrains their ability to evaluate the capabilities of modern multimodal agents designed for GUI-based reasoning. To address this gap, we introduce DashboardQA, the first benchmark explicitly designed to assess how vision-language GUI agents comprehend and interact with real-world dashboards. The benchmark includes 112 interactive dashboards from Tableau Public and 405 question-answer pairs with interactive dashboards spanning five categories: multiple-choice, factoid, hypothetical, multi-dashboard, and conversational. By assessing a variety of leading closed- and open-source GUI agents, our analysis reveals their key limitations, particularly in grounding dashboard elements, planning interaction trajectories, and performing reasoning. Our findings indicate that interactive dashboard reasoning is a challenging task overall for all the VLMs evaluated. Even the top-performing agents struggle; for instance, the best agent based on Gemini-Pro-2.5 achieves only 38.69% accuracy, while the OpenAI CUA agent reaches just 22.69%, demonstrating the benchmark's significant difficulty. We release DashboardQA at https://github.com/vis-nlp/DashboardQA |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_17398 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards Kartha, Aaryaman Masry, Ahmed Islam, Mohammed Saidul Lang, Thinh Rahman, Shadikur Mahbub, Ridwan Rahman, Mizanur Ahmed, Mahir Parvez, Md Rizwan Hoque, Enamul Joty, Shafiq Computation and Language Dashboards are powerful visualization tools for data-driven decision-making, integrating multiple interactive views that allow users to explore, filter, and navigate data. Unlike static charts, dashboards support rich interactivity, which is essential for uncovering insights in real-world analytical workflows. However, existing question-answering benchmarks for data visualizations largely overlook this interactivity, focusing instead on static charts. This limitation severely constrains their ability to evaluate the capabilities of modern multimodal agents designed for GUI-based reasoning. To address this gap, we introduce DashboardQA, the first benchmark explicitly designed to assess how vision-language GUI agents comprehend and interact with real-world dashboards. The benchmark includes 112 interactive dashboards from Tableau Public and 405 question-answer pairs with interactive dashboards spanning five categories: multiple-choice, factoid, hypothetical, multi-dashboard, and conversational. By assessing a variety of leading closed- and open-source GUI agents, our analysis reveals their key limitations, particularly in grounding dashboard elements, planning interaction trajectories, and performing reasoning. Our findings indicate that interactive dashboard reasoning is a challenging task overall for all the VLMs evaluated. Even the top-performing agents struggle; for instance, the best agent based on Gemini-Pro-2.5 achieves only 38.69% accuracy, while the OpenAI CUA agent reaches just 22.69%, demonstrating the benchmark's significant difficulty. We release DashboardQA at https://github.com/vis-nlp/DashboardQA |
| title | DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2508.17398 |