Experimental Evaluation of Static Image Sub-Region-Based Search Models Using CLIP
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913884854550528 |
|---|---|
| author | Jäckl, Bastian Kloda, Vojtěch Keim, Daniel A. Lokoč, Jakub |
| author_facet | Jäckl, Bastian Kloda, Vojtěch Keim, Daniel A. Lokoč, Jakub |
| contents | Advances in multimodal text-image models have enabled effective text-based querying in extensive image collections. While these models show convincing performance for everyday life scenes, querying in highly homogeneous, specialized domains remains challenging. The primary problem is that users can often provide only vague textual descriptions as they lack expert knowledge to discriminate between homogenous entities. This work investigates whether adding location-based prompts to complement these vague text queries can enhance retrieval performance. Specifically, we collected a dataset of 741 human annotations, each containing short and long textual descriptions and bounding boxes indicating regions of interest in challenging underwater scenes. Using these annotations, we evaluate the performance of CLIP when queried on various static sub-regions of images compared to the full image. Our results show that both a simple 3-by-3 partitioning and a 5-grid overlap significantly improve retrieval effectiveness and remain robust to perturbations of the annotation box. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_06938 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Experimental Evaluation of Static Image Sub-Region-Based Search Models Using CLIP Jäckl, Bastian Kloda, Vojtěch Keim, Daniel A. Lokoč, Jakub Multimedia Computer Vision and Pattern Recognition 68U10 H.3.3; I.4.10; H.2.8 Advances in multimodal text-image models have enabled effective text-based querying in extensive image collections. While these models show convincing performance for everyday life scenes, querying in highly homogeneous, specialized domains remains challenging. The primary problem is that users can often provide only vague textual descriptions as they lack expert knowledge to discriminate between homogenous entities. This work investigates whether adding location-based prompts to complement these vague text queries can enhance retrieval performance. Specifically, we collected a dataset of 741 human annotations, each containing short and long textual descriptions and bounding boxes indicating regions of interest in challenging underwater scenes. Using these annotations, we evaluate the performance of CLIP when queried on various static sub-regions of images compared to the full image. Our results show that both a simple 3-by-3 partitioning and a 5-grid overlap significantly improve retrieval effectiveness and remain robust to perturbations of the annotation box. |
| title | Experimental Evaluation of Static Image Sub-Region-Based Search Models Using CLIP |
| topic | Multimedia Computer Vision and Pattern Recognition 68U10 H.3.3; I.4.10; H.2.8 |
| url | https://arxiv.org/abs/2506.06938 |