Dynamic Sub-region Search in Homogeneous Collections Using CLIP
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908404208893952 |
|---|---|
| author | Jäckl, Bastian Kloda, Vojtěch Keim, Daniel A. Lokoč, Jakub |
| author_facet | Jäckl, Bastian Kloda, Vojtěch Keim, Daniel A. Lokoč, Jakub |
| contents | Querying with text-image-based search engines in highly homogeneous domain-specific image collections is challenging for users, as they often struggle to provide descriptive text queries. For example, in an underwater domain, users can usually characterize entities only with abstract labels, such as corals and fish, which leads to low recall rates. Our work investigates whether recall can be improved by supplementing text queries with position information. Specifically, we explore dynamic image partitioning approaches that divide candidates into semantically meaningful regions of interest. Instead of querying entire images, users can specify regions they recognize. This enables the use of position constraints while preserving the semantic capabilities of multimodal models. We introduce and evaluate strategies for integrating position constraints into semantic search models and compare them against static partitioning approaches. Our evaluation highlights both the potential and the limitations of sub-region-based search methods using dynamic partitioning. Dynamic search models achieve up to double the retrieval performance compared to static partitioning approaches but are highly sensitive to perturbations in the specified query positions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_09506 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dynamic Sub-region Search in Homogeneous Collections Using CLIP Jäckl, Bastian Kloda, Vojtěch Keim, Daniel A. Lokoč, Jakub Multimedia 68U10 H.3.3; I.4.10; H.2.8 Querying with text-image-based search engines in highly homogeneous domain-specific image collections is challenging for users, as they often struggle to provide descriptive text queries. For example, in an underwater domain, users can usually characterize entities only with abstract labels, such as corals and fish, which leads to low recall rates. Our work investigates whether recall can be improved by supplementing text queries with position information. Specifically, we explore dynamic image partitioning approaches that divide candidates into semantically meaningful regions of interest. Instead of querying entire images, users can specify regions they recognize. This enables the use of position constraints while preserving the semantic capabilities of multimodal models. We introduce and evaluate strategies for integrating position constraints into semantic search models and compare them against static partitioning approaches. Our evaluation highlights both the potential and the limitations of sub-region-based search methods using dynamic partitioning. Dynamic search models achieve up to double the retrieval performance compared to static partitioning approaches but are highly sensitive to perturbations in the specified query positions. |
| title | Dynamic Sub-region Search in Homogeneous Collections Using CLIP |
| topic | Multimedia 68U10 H.3.3; I.4.10; H.2.8 |
| url | https://arxiv.org/abs/2506.09506 |