BrowseComp-$V^3$: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917290150199296 |
|---|---|
| author | Zhang, Huanyao Zhou, Jiepeng Li, Bo Zhou, Bowen Shan, Yanzhe Lu, Haishan Cao, Zhiyong Chen, Jiaoyang Han, Yuqian Sheng, Zinan Tao, Zhengwei Liang, Hao Wu, Jialong Shi, Yang He, Yuanpeng Lin, Jiaye Zhang, Qintong Yan, Guochen Zhao, Runhao Li, Zhengpin Yu, Xiaohan Mei, Lang Chen, Chong Zhang, Wentao Cui, Bin |
| author_facet | Zhang, Huanyao Zhou, Jiepeng Li, Bo Zhou, Bowen Shan, Yanzhe Lu, Haishan Cao, Zhiyong Chen, Jiaoyang Han, Yuqian Sheng, Zinan Tao, Zhengwei Liang, Hao Wu, Jialong Shi, Yang He, Yuanpeng Lin, Jiaye Zhang, Qintong Yan, Guochen Zhao, Runhao Li, Zhengpin Yu, Xiaohan Mei, Lang Chen, Chong Zhang, Wentao Cui, Bin |
| contents | Multimodal large language models (MLLMs), equipped with increasingly advanced planning and tool-use capabilities, are evolving into autonomous agents capable of performing multimodal web browsing and deep search in open-world environments. However, existing benchmarks for multimodal browsing remain limited in task complexity, evidence accessibility, and evaluation granularity, hindering comprehensive and reproducible assessments of deep search capabilities. To address these limitations, we introduce BrowseComp-$V^3$, a novel benchmark consisting of 300 carefully curated and challenging questions spanning diverse domains. The benchmark emphasizes deep, multi-level, and cross-modal multi-hop reasoning, where critical evidence is interleaved across textual and visual modalities within and across web pages. All supporting evidence is strictly required to be publicly searchable, ensuring fairness and reproducibility. Beyond final-answer accuracy, we incorporate an expert-validated, subgoal-driven process evaluation mechanism that enables fine-grained analysis of intermediate reasoning behaviors and systematic characterization of capability boundaries. In addition, we propose OmniSeeker, a unified multimodal browsing agent framework integrating diverse web search and visual perception tools. Comprehensive experiments demonstrate that even state-of-the-art models achieve only 36% accuracy on our benchmark, revealing critical bottlenecks in multimodal information integration and fine-grained perception. Our results highlight a fundamental gap between current model capabilities and robust multimodal deep search in real-world settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_12876 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | BrowseComp-$V^3$: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents Zhang, Huanyao Zhou, Jiepeng Li, Bo Zhou, Bowen Shan, Yanzhe Lu, Haishan Cao, Zhiyong Chen, Jiaoyang Han, Yuqian Sheng, Zinan Tao, Zhengwei Liang, Hao Wu, Jialong Shi, Yang He, Yuanpeng Lin, Jiaye Zhang, Qintong Yan, Guochen Zhao, Runhao Li, Zhengpin Yu, Xiaohan Mei, Lang Chen, Chong Zhang, Wentao Cui, Bin Artificial Intelligence Multimodal large language models (MLLMs), equipped with increasingly advanced planning and tool-use capabilities, are evolving into autonomous agents capable of performing multimodal web browsing and deep search in open-world environments. However, existing benchmarks for multimodal browsing remain limited in task complexity, evidence accessibility, and evaluation granularity, hindering comprehensive and reproducible assessments of deep search capabilities. To address these limitations, we introduce BrowseComp-$V^3$, a novel benchmark consisting of 300 carefully curated and challenging questions spanning diverse domains. The benchmark emphasizes deep, multi-level, and cross-modal multi-hop reasoning, where critical evidence is interleaved across textual and visual modalities within and across web pages. All supporting evidence is strictly required to be publicly searchable, ensuring fairness and reproducibility. Beyond final-answer accuracy, we incorporate an expert-validated, subgoal-driven process evaluation mechanism that enables fine-grained analysis of intermediate reasoning behaviors and systematic characterization of capability boundaries. In addition, we propose OmniSeeker, a unified multimodal browsing agent framework integrating diverse web search and visual perception tools. Comprehensive experiments demonstrate that even state-of-the-art models achieve only 36% accuracy on our benchmark, revealing critical bottlenecks in multimodal information integration and fine-grained perception. Our results highlight a fundamental gap between current model capabilities and robust multimodal deep search in real-world settings. |
| title | BrowseComp-$V^3$: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2602.12876 |