Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910057288957952 |
|---|---|
| author | Shabtay, Nimrod Kimhi, Moshe Spector, Artem Haray, Sivan Rivlin, Ehud Baskin, Chaim Giryes, Raja Schwartz, Eli |
| author_facet | Shabtay, Nimrod Kimhi, Moshe Spector, Artem Haray, Sivan Rivlin, Ehud Baskin, Chaim Giryes, Raja Schwartz, Eli |
| contents | Vision-language models (VLMs) typically process images at a native high-resolution, forcing a trade-off between accuracy and computational efficiency: high-resolution inputs capture fine details but incur significant computational costs, while low-resolution inputs advocate for efficiency, they potentially miss critical visual information, like small text. We present AwaRes, a spatial-on-demand framework that resolves this accuracy-efficiency trade-off by operating on a low-resolution global view and using tool-calling to retrieve only high-resolution segments needed for a given query. We construct supervised data automatically: a judge compares low- vs.\ high-resolution answers to label whether cropping is needed, and an oracle grounding model localizes the evidence for the correct answer, which we map to a discrete crop set to form multi-turn tool-use trajectories. We train our framework with cold-start SFT followed by multi-turn GRPO with a composite reward that combines semantic answer correctness with explicit crop-cost penalties. Project page: https://nimrodshabtay.github.io/AwaRes |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_16932 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs Shabtay, Nimrod Kimhi, Moshe Spector, Artem Haray, Sivan Rivlin, Ehud Baskin, Chaim Giryes, Raja Schwartz, Eli Computer Vision and Pattern Recognition Artificial Intelligence Vision-language models (VLMs) typically process images at a native high-resolution, forcing a trade-off between accuracy and computational efficiency: high-resolution inputs capture fine details but incur significant computational costs, while low-resolution inputs advocate for efficiency, they potentially miss critical visual information, like small text. We present AwaRes, a spatial-on-demand framework that resolves this accuracy-efficiency trade-off by operating on a low-resolution global view and using tool-calling to retrieve only high-resolution segments needed for a given query. We construct supervised data automatically: a judge compares low- vs.\ high-resolution answers to label whether cropping is needed, and an oracle grounding model localizes the evidence for the correct answer, which we map to a discrete crop set to form multi-turn tool-use trajectories. We train our framework with cold-start SFT followed by multi-turn GRPO with a composite reward that combines semantic answer correctness with explicit crop-cost penalties. Project page: https://nimrodshabtay.github.io/AwaRes |
| title | Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2603.16932 |