ITEm: Unsupervised Image-Text Embedding Learning for eCommerce
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909119571558400 |
|---|---|
| author | Liao, Baohao Kozielski, Michael Hewavitharana, Sanjika Yuan, Jiangbo Khadivi, Shahram Lancewicki, Tomer |
| author_facet | Liao, Baohao Kozielski, Michael Hewavitharana, Sanjika Yuan, Jiangbo Khadivi, Shahram Lancewicki, Tomer |
| contents | Product embedding serves as a cornerstone for a wide range of applications in eCommerce. The product embedding learned from multiple modalities shows significant improvement over that from a single modality, since different modalities provide complementary information. However, some modalities are more informatively dominant than others. How to teach a model to learn embedding from different modalities without neglecting information from the less dominant modality is challenging. We present an image-text embedding model (ITEm), an unsupervised learning method that is designed to better attend to image and text modalities. We extend BERT by (1) learning an embedding from text and image without knowing the regions of interest; (2) training a global representation to predict masked words and to construct masked image patches without their individual representations. We evaluate the pre-trained ITEm on two tasks: the search for extremely similar products and the prediction of product categories, showing substantial gains compared to strong baseline models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_02084 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | ITEm: Unsupervised Image-Text Embedding Learning for eCommerce Liao, Baohao Kozielski, Michael Hewavitharana, Sanjika Yuan, Jiangbo Khadivi, Shahram Lancewicki, Tomer Computer Vision and Pattern Recognition Computation and Language Information Retrieval Product embedding serves as a cornerstone for a wide range of applications in eCommerce. The product embedding learned from multiple modalities shows significant improvement over that from a single modality, since different modalities provide complementary information. However, some modalities are more informatively dominant than others. How to teach a model to learn embedding from different modalities without neglecting information from the less dominant modality is challenging. We present an image-text embedding model (ITEm), an unsupervised learning method that is designed to better attend to image and text modalities. We extend BERT by (1) learning an embedding from text and image without knowing the regions of interest; (2) training a global representation to predict masked words and to construct masked image patches without their individual representations. We evaluate the pre-trained ITEm on two tasks: the search for extremely similar products and the prediction of product categories, showing substantial gains compared to strong baseline models. |
| title | ITEm: Unsupervised Image-Text Embedding Learning for eCommerce |
| topic | Computer Vision and Pattern Recognition Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2311.02084 |