What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913023482920960 |
|---|---|
| author | Choi, Dasol Son, Guijin Lee, Hanwool Kim, Minhyuk Ko, Hyunwoo Lim, Teabin Eungyeol, Ahn Kim, Jungwhan Hong, Seunghyeok Song, Youngsook |
| author_facet | Choi, Dasol Son, Guijin Lee, Hanwool Kim, Minhyuk Ko, Hyunwoo Lim, Teabin Eungyeol, Ahn Kim, Jungwhan Hong, Seunghyeok Song, Youngsook |
| contents | Current vision-language benchmarks predominantly feature well-structured questions with clear, explicit prompts. However, real user queries are often informal and underspecified. Users naturally leave much unsaid, relying on images to convey context. We introduce HAERAE-Vision, a benchmark of 653 real-world visual questions from Korean online communities (0.76% survival from 86K candidates), each paired with an explicit rewrite, yielding 1,306 query variants in total. Evaluating 39 VLMs, we find that even state-of-the-art models (GPT-5, Gemini 2.5 Pro) achieve under 50% on the original queries. Crucially, query explicitation alone yields 8 to 22 point improvements, with smaller models benefiting most. We further show that even with web search, under-specified queries underperform explicit queries without search, revealing that current retrieval cannot compensate for what users leave unsaid. Our findings demonstrate that a substantial portion of VLM difficulty stem from natural query under-specification instead of model capability, highlighting a critical gap between benchmark evaluation and real-world deployment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_06165 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models Choi, Dasol Son, Guijin Lee, Hanwool Kim, Minhyuk Ko, Hyunwoo Lim, Teabin Eungyeol, Ahn Kim, Jungwhan Hong, Seunghyeok Song, Youngsook Computer Vision and Pattern Recognition Artificial Intelligence Current vision-language benchmarks predominantly feature well-structured questions with clear, explicit prompts. However, real user queries are often informal and underspecified. Users naturally leave much unsaid, relying on images to convey context. We introduce HAERAE-Vision, a benchmark of 653 real-world visual questions from Korean online communities (0.76% survival from 86K candidates), each paired with an explicit rewrite, yielding 1,306 query variants in total. Evaluating 39 VLMs, we find that even state-of-the-art models (GPT-5, Gemini 2.5 Pro) achieve under 50% on the original queries. Crucially, query explicitation alone yields 8 to 22 point improvements, with smaller models benefiting most. We further show that even with web search, under-specified queries underperform explicit queries without search, revealing that current retrieval cannot compensate for what users leave unsaid. Our findings demonstrate that a substantial portion of VLM difficulty stem from natural query under-specification instead of model capability, highlighting a critical gap between benchmark evaluation and real-world deployment. |
| title | What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2601.06165 |