Gorde:
| Egile Nagusiak: | , , , , , |
|---|---|
| Formatua: | Preprint |
| Argitaratua: |
2026
|
| Gaiak: | |
| Sarrera elektronikoa: | https://arxiv.org/abs/2602.13402 |
| Etiketak: |
Etiketa erantsi
Etiketarik gabe, Izan zaitez lehena erregistro honi etiketa jartzen!
|
| _version_ | 1866914329219039232 |
|---|---|
| author | Dravilas, Ioannis Kapetangeorgis, Ioannis Latsoudis, Anastasios McCarthy, Conor Marcelino, Gonçalo Worring, Marcel |
| author_facet | Dravilas, Ioannis Kapetangeorgis, Ioannis Latsoudis, Anastasios McCarthy, Conor Marcelino, Gonçalo Worring, Marcel |
| contents | Composed Image Retrieval (CIR) allows users to search for images by combining a reference image with a text prompt that describes desired modifications. While vision-language models like CLIP have popularized this task by embedding multiple modalities into a joint space, developers still lack tools that reveal how these multimodal prompts interact with embedding spaces and why small wording changes can dramatically alter the results. We present InfoCIR, a visual analytics system that closes this gap by coupling retrieval, explainability, and prompt engineering in a single, interactive dashboard. InfoCIR integrates a state-of-the-art CIR back-end (SEARLE arXiv:2303.15247) with a six-panel interface that (i) lets users compose image + text queries, (ii) projects the top-k results into a low-dimensional space using Uniform Manifold Approximation and Projection (UMAP) for spatial reasoning, (iii) overlays similarity-based saliency maps and gradient-derived token-attribution bars for local explanation, and (iv) employs an LLM-powered prompt enhancer that generates counterfactual variants and visualizes how these changes affect the ranking of user-selected target images. A modular architecture built on Plotly-Dash allows new models, datasets, and attribution methods to be plugged in with minimal effort. We argue that InfoCIR helps diagnose retrieval failures, guides prompt enhancement, and accelerates insight generation during model development. All source code allowing for a reproducible demo is available at https://github.com/giannhskp/InfoCIR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_13402 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | InfoCIR: Multimedia Analysis for Composed Image Retrieval Dravilas, Ioannis Kapetangeorgis, Ioannis Latsoudis, Anastasios McCarthy, Conor Marcelino, Gonçalo Worring, Marcel Human-Computer Interaction Information Retrieval Multimedia Composed Image Retrieval (CIR) allows users to search for images by combining a reference image with a text prompt that describes desired modifications. While vision-language models like CLIP have popularized this task by embedding multiple modalities into a joint space, developers still lack tools that reveal how these multimodal prompts interact with embedding spaces and why small wording changes can dramatically alter the results. We present InfoCIR, a visual analytics system that closes this gap by coupling retrieval, explainability, and prompt engineering in a single, interactive dashboard. InfoCIR integrates a state-of-the-art CIR back-end (SEARLE arXiv:2303.15247) with a six-panel interface that (i) lets users compose image + text queries, (ii) projects the top-k results into a low-dimensional space using Uniform Manifold Approximation and Projection (UMAP) for spatial reasoning, (iii) overlays similarity-based saliency maps and gradient-derived token-attribution bars for local explanation, and (iv) employs an LLM-powered prompt enhancer that generates counterfactual variants and visualizes how these changes affect the ranking of user-selected target images. A modular architecture built on Plotly-Dash allows new models, datasets, and attribution methods to be plugged in with minimal effort. We argue that InfoCIR helps diagnose retrieval failures, guides prompt enhancement, and accelerates insight generation during model development. All source code allowing for a reproducible demo is available at https://github.com/giannhskp/InfoCIR. |
| title | InfoCIR: Multimedia Analysis for Composed Image Retrieval |
| topic | Human-Computer Interaction Information Retrieval Multimedia |
| url | https://arxiv.org/abs/2602.13402 |