LocateBench: Evaluating the Locating Ability of Vision Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917818193149952 |
|---|---|
| author | Chiang, Ting-Rui Robinson, Joshua Yu, Xinyan Velocity Yogatama, Dani |
| author_facet | Chiang, Ting-Rui Robinson, Joshua Yu, Xinyan Velocity Yogatama, Dani |
| contents | The ability to locate an object in an image according to natural language instructions is crucial for many real-world applications. In this work we propose LocateBench, a high-quality benchmark dedicated to evaluating this ability. We experiment with multiple prompting approaches, and measure the accuracy of several large vision language models. We find that even the accuracy of the strongest model, GPT-4o, lags behind human accuracy by more than 10%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_19808 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LocateBench: Evaluating the Locating Ability of Vision Language Models Chiang, Ting-Rui Robinson, Joshua Yu, Xinyan Velocity Yogatama, Dani Computer Vision and Pattern Recognition Artificial Intelligence The ability to locate an object in an image according to natural language instructions is crucial for many real-world applications. In this work we propose LocateBench, a high-quality benchmark dedicated to evaluating this ability. We experiment with multiple prompting approaches, and measure the accuracy of several large vision language models. We find that even the accuracy of the strongest model, GPT-4o, lags behind human accuracy by more than 10%. |
| title | LocateBench: Evaluating the Locating Ability of Vision Language Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2410.19808 |