SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918419164561408 |
|---|---|
| author | Ai, Kuangshi Miao, Haichao Tang, Kaiyuan Gorski, Nathaniel Sun, Jianxin Liu, Guoxi Ingolfsson, Helgi I. Lenz, David Guo, Hanqi Yu, Hongfeng Leburu, Teja Molash, Michael Wang, Bei Peterka, Tom Wang, Chaoli Liu, Shusen |
| author_facet | Ai, Kuangshi Miao, Haichao Tang, Kaiyuan Gorski, Nathaniel Sun, Jianxin Liu, Guoxi Ingolfsson, Helgi I. Lenz, David Guo, Hanqi Yu, Hongfeng Leburu, Teja Molash, Michael Wang, Bei Peterka, Tom Wang, Chaoli Liu, Shusen |
| contents | Recent advances in large language models (LLMs) have enabled agentic systems that translate natural language intent into executable scientific visualization (SciVis) tasks. Despite rapid progress, the community lacks a principled and reproducible benchmark for evaluating these emerging SciVis agents in realistic, multi-step analysis settings. We present SciVisAgentBench, a comprehensive and extensible benchmark for evaluating scientific data analysis and visualization agents. Our benchmark is grounded in a structured taxonomy spanning four dimensions: application domain, data type, complexity level, and visualization operation. It currently comprises 108 expert-crafted cases covering diverse SciVis scenarios. To enable reliable assessment, we introduce a multimodal outcome-centric evaluation pipeline that combines LLM-based judging with deterministic evaluators, including image-based metrics, code checkers, rule-based verifiers, and case-specific evaluators. We also conduct a validity study with 12 SciVis experts to examine the agreement between human and LLM judges. Using this framework, we evaluate representative SciVis agents and general-purpose coding agents to establish initial baselines and reveal capability gaps. SciVisAgentBench is designed as a living benchmark to support systematic comparison, diagnose failure modes, and drive progress in agentic SciVis. The benchmark is available at https://scivisagentbench.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_29139 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents Ai, Kuangshi Miao, Haichao Tang, Kaiyuan Gorski, Nathaniel Sun, Jianxin Liu, Guoxi Ingolfsson, Helgi I. Lenz, David Guo, Hanqi Yu, Hongfeng Leburu, Teja Molash, Michael Wang, Bei Peterka, Tom Wang, Chaoli Liu, Shusen Artificial Intelligence Graphics Human-Computer Interaction Recent advances in large language models (LLMs) have enabled agentic systems that translate natural language intent into executable scientific visualization (SciVis) tasks. Despite rapid progress, the community lacks a principled and reproducible benchmark for evaluating these emerging SciVis agents in realistic, multi-step analysis settings. We present SciVisAgentBench, a comprehensive and extensible benchmark for evaluating scientific data analysis and visualization agents. Our benchmark is grounded in a structured taxonomy spanning four dimensions: application domain, data type, complexity level, and visualization operation. It currently comprises 108 expert-crafted cases covering diverse SciVis scenarios. To enable reliable assessment, we introduce a multimodal outcome-centric evaluation pipeline that combines LLM-based judging with deterministic evaluators, including image-based metrics, code checkers, rule-based verifiers, and case-specific evaluators. We also conduct a validity study with 12 SciVis experts to examine the agreement between human and LLM judges. Using this framework, we evaluate representative SciVis agents and general-purpose coding agents to establish initial baselines and reveal capability gaps. SciVisAgentBench is designed as a living benchmark to support systematic comparison, diagnose failure modes, and drive progress in agentic SciVis. The benchmark is available at https://scivisagentbench.github.io/. |
| title | SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents |
| topic | Artificial Intelligence Graphics Human-Computer Interaction |
| url | https://arxiv.org/abs/2603.29139 |