AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarios
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914362992623616 |
|---|---|
| author | Su, Zhaochen Gao, Jincheng Guo, Hangyu Liu, Zhenhua Zhang, Lueyang Geng, Xinyu Huang, Shijue Xia, Peng Jiang, Guanyu Wang, Cheng Zhang, Yue Fung, Yi R. He, Junxian |
| author_facet | Su, Zhaochen Gao, Jincheng Guo, Hangyu Liu, Zhenhua Zhang, Lueyang Geng, Xinyu Huang, Shijue Xia, Peng Jiang, Guanyu Wang, Cheng Zhang, Yue Fung, Yi R. He, Junxian |
| contents | Real-world multimodal agents solve multi-step workflows grounded in visual evidence. For example, an agent can troubleshoot a device by linking a wiring photo to a schematic and validating the fix with online documentation, or plan a trip by interpreting a transit map and checking schedules under routing constraints. However, existing multimodal benchmarks mainly evaluate single-turn visual reasoning or specific tool skills, and they do not fully capture the realism, visual subtlety, and long-horizon tool use that practical agents require. We introduce AgentVista, a benchmark for generalist multimodal agents that spans 25 sub-domains across 7 categories, pairing realistic and detail-rich visual scenarios with natural hybrid tool use. Tasks require long-horizon tool interactions across modalities, including web search, image search, page navigation, and code-based operations for both image processing and general programming. Comprehensive evaluation of state-of-the-art models exposes significant gaps in their ability to carry out long-horizon multimodal tool use. Even the best model in our evaluation, Gemini-3-Pro with tools, achieves only 27.3% overall accuracy, and hard instances can require more than 25 tool-calling turns. We expect AgentVista to accelerate the development of more capable and reliable multimodal agents for realistic and ultra-challenging problem solving. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_23166 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarios Su, Zhaochen Gao, Jincheng Guo, Hangyu Liu, Zhenhua Zhang, Lueyang Geng, Xinyu Huang, Shijue Xia, Peng Jiang, Guanyu Wang, Cheng Zhang, Yue Fung, Yi R. He, Junxian Computer Vision and Pattern Recognition Real-world multimodal agents solve multi-step workflows grounded in visual evidence. For example, an agent can troubleshoot a device by linking a wiring photo to a schematic and validating the fix with online documentation, or plan a trip by interpreting a transit map and checking schedules under routing constraints. However, existing multimodal benchmarks mainly evaluate single-turn visual reasoning or specific tool skills, and they do not fully capture the realism, visual subtlety, and long-horizon tool use that practical agents require. We introduce AgentVista, a benchmark for generalist multimodal agents that spans 25 sub-domains across 7 categories, pairing realistic and detail-rich visual scenarios with natural hybrid tool use. Tasks require long-horizon tool interactions across modalities, including web search, image search, page navigation, and code-based operations for both image processing and general programming. Comprehensive evaluation of state-of-the-art models exposes significant gaps in their ability to carry out long-horizon multimodal tool use. Even the best model in our evaluation, Gemini-3-Pro with tools, achieves only 27.3% overall accuracy, and hard instances can require more than 25 tool-calling turns. We expect AgentVista to accelerate the development of more capable and reliable multimodal agents for realistic and ultra-challenging problem solving. |
| title | AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarios |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2602.23166 |