Composed Image Retrieval for Training-Free Domain Conversion
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915048056684544 |
|---|---|
| author | Efthymiadis, Nikos Psomas, Bill Laskar, Zakaria Karantzalos, Konstantinos Avrithis, Yannis Chum, Ondřej Tolias, Giorgos |
| author_facet | Efthymiadis, Nikos Psomas, Bill Laskar, Zakaria Karantzalos, Konstantinos Avrithis, Yannis Chum, Ondřej Tolias, Giorgos |
| contents | This work addresses composed image retrieval in the context of domain conversion, where the content of a query image is retrieved in the domain specified by the query text. We show that a strong vision-language model provides sufficient descriptive power without additional training. The query image is mapped to the text input space using textual inversion. Unlike common practice that invert in the continuous space of text tokens, we use the discrete word space via a nearest-neighbor search in a text vocabulary. With this inversion, the image is softly mapped across the vocabulary and is made more robust using retrieval-based augmentation. Database images are retrieved by a weighted ensemble of text queries combining mapped words with the domain text. Our method outperforms prior art by a large margin on standard and newly introduced benchmarks. Code: https://github.com/NikosEfth/freedom |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_03297 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Composed Image Retrieval for Training-Free Domain Conversion Efthymiadis, Nikos Psomas, Bill Laskar, Zakaria Karantzalos, Konstantinos Avrithis, Yannis Chum, Ondřej Tolias, Giorgos Computer Vision and Pattern Recognition This work addresses composed image retrieval in the context of domain conversion, where the content of a query image is retrieved in the domain specified by the query text. We show that a strong vision-language model provides sufficient descriptive power without additional training. The query image is mapped to the text input space using textual inversion. Unlike common practice that invert in the continuous space of text tokens, we use the discrete word space via a nearest-neighbor search in a text vocabulary. With this inversion, the image is softly mapped across the vocabulary and is made more robust using retrieval-based augmentation. Database images are retrieved by a weighted ensemble of text queries combining mapped words with the domain text. Our method outperforms prior art by a large margin on standard and newly introduced benchmarks. Code: https://github.com/NikosEfth/freedom |
| title | Composed Image Retrieval for Training-Free Domain Conversion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.03297 |