Captions Are Worth a Thousand Words: Enhancing Product Retrieval with Pretrained Image-to-Text Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916123500347392 |
|---|---|
| author | Tang, Jason McGoldrick, Garrin Al-Ghossein, Marie Chen, Ching-Wei |
| author_facet | Tang, Jason McGoldrick, Garrin Al-Ghossein, Marie Chen, Ching-Wei |
| contents | This paper explores the usage of multimodal image-to-text models to enhance text-based item retrieval. We propose utilizing pre-trained image captioning and tagging models, such as instructBLIP and CLIP, to generate text-based product descriptions which are combined with existing text descriptions. Our work is particularly impactful for smaller eCommerce businesses who are unable to maintain the high-quality text descriptions necessary to effectively perform item retrieval for search and recommendation use cases. We evaluate the searchability of ground-truth text, image-generated text, and combinations of both texts on several subsets of Amazon's publicly available ESCI dataset. The results demonstrate the dual capability of our proposed models to enhance the retrieval of existing text and generate highly-searchable standalone descriptions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_08532 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Captions Are Worth a Thousand Words: Enhancing Product Retrieval with Pretrained Image-to-Text Models Tang, Jason McGoldrick, Garrin Al-Ghossein, Marie Chen, Ching-Wei Information Retrieval This paper explores the usage of multimodal image-to-text models to enhance text-based item retrieval. We propose utilizing pre-trained image captioning and tagging models, such as instructBLIP and CLIP, to generate text-based product descriptions which are combined with existing text descriptions. Our work is particularly impactful for smaller eCommerce businesses who are unable to maintain the high-quality text descriptions necessary to effectively perform item retrieval for search and recommendation use cases. We evaluate the searchability of ground-truth text, image-generated text, and combinations of both texts on several subsets of Amazon's publicly available ESCI dataset. The results demonstrate the dual capability of our proposed models to enhance the retrieval of existing text and generate highly-searchable standalone descriptions. |
| title | Captions Are Worth a Thousand Words: Enhancing Product Retrieval with Pretrained Image-to-Text Models |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2402.08532 |