OWLViz: An Open-World Benchmark for Visual Question Answering
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908472115724288 |
|---|---|
| author | Nguyen, Thuy Nguyen, Dang Nguyen, Hoang Luong, Thuan Dang, Long Hoang Lai, Viet Dac |
| author_facet | Nguyen, Thuy Nguyen, Dang Nguyen, Hoang Luong, Thuan Dang, Long Hoang Lai, Viet Dac |
| contents | We present a challenging benchmark for the Open WorLd VISual question answering (OWLViz) task. OWLViz presents concise, unambiguous queries that require integrating multiple capabilities, including visual understanding, web exploration, and specialized tool usage. While humans achieve 69.2% accuracy on these intuitive tasks, even state-of-the-art VLMs struggle, with the best model, Gemini 2.0, achieving only 26.6% accuracy. Current agentic VLMs, which rely on limited vision and vision-language models as tools, perform even worse. This performance gap reveals significant limitations in multimodal systems' ability to select appropriate tools and execute complex reasoning sequences, establishing new directions for advancing practical AI research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_07631 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OWLViz: An Open-World Benchmark for Visual Question Answering Nguyen, Thuy Nguyen, Dang Nguyen, Hoang Luong, Thuan Dang, Long Hoang Lai, Viet Dac Machine Learning Computation and Language We present a challenging benchmark for the Open WorLd VISual question answering (OWLViz) task. OWLViz presents concise, unambiguous queries that require integrating multiple capabilities, including visual understanding, web exploration, and specialized tool usage. While humans achieve 69.2% accuracy on these intuitive tasks, even state-of-the-art VLMs struggle, with the best model, Gemini 2.0, achieving only 26.6% accuracy. Current agentic VLMs, which rely on limited vision and vision-language models as tools, perform even worse. This performance gap reveals significant limitations in multimodal systems' ability to select appropriate tools and execute complex reasoning sequences, establishing new directions for advancing practical AI research. |
| title | OWLViz: An Open-World Benchmark for Visual Question Answering |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2503.07631 |