Semantic-enhanced Modality-asymmetric Retrieval for Online E-commerce Search

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Zhigong, Ding, Ning, Fan, Xiaochuan, Shang, Yue, Qiu, Yiming, Zhuo, Jingwei, Ge, Zhiwei, Wang, Songlin, Liu, Lin, Xu, Sulong, Zhang, Han
Format:	Preprint
Published:	2025
Subjects:	Information Retrieval
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866911021687373824
author	Zhou, Zhigong Ding, Ning Fan, Xiaochuan Shang, Yue Qiu, Yiming Zhuo, Jingwei Ge, Zhiwei Wang, Songlin Liu, Lin Xu, Sulong Zhang, Han
author_facet	Zhou, Zhigong Ding, Ning Fan, Xiaochuan Shang, Yue Qiu, Yiming Zhuo, Jingwei Ge, Zhiwei Wang, Songlin Liu, Lin Xu, Sulong Zhang, Han
contents	Semantic retrieval, which retrieves semantically matched items given a textual query, has been an essential component to enhance system effectiveness in e-commerce search. In this paper, we study the multimodal retrieval problem, where the visual information (e.g, image) of item is leveraged as supplementary of textual information to enrich item representation and further improve retrieval performance. Though learning from cross-modality data has been studied extensively in tasks such as visual question answering or media summarization, multimodal retrieval remains a non-trivial and unsolved problem especially in the asymmetric scenario where the query is unimodal while the item is multimodal. In this paper, we propose a novel model named SMAR, which stands for Semantic-enhanced Modality-Asymmetric Retrieval, to tackle the problem of modality fusion and alignment in this kind of asymmetric scenario. Extensive experimental results on an industrial dataset show that the proposed model outperforms baseline models significantly in retrieval accuracy. We have open sourced our industrial dataset for the sake of reproducibility and future research works.
format	Preprint
id	arxiv_https___arxiv_org_abs_2506_20330
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Semantic-enhanced Modality-asymmetric Retrieval for Online E-commerce Search Zhou, Zhigong Ding, Ning Fan, Xiaochuan Shang, Yue Qiu, Yiming Zhuo, Jingwei Ge, Zhiwei Wang, Songlin Liu, Lin Xu, Sulong Zhang, Han Information Retrieval Semantic retrieval, which retrieves semantically matched items given a textual query, has been an essential component to enhance system effectiveness in e-commerce search. In this paper, we study the multimodal retrieval problem, where the visual information (e.g, image) of item is leveraged as supplementary of textual information to enrich item representation and further improve retrieval performance. Though learning from cross-modality data has been studied extensively in tasks such as visual question answering or media summarization, multimodal retrieval remains a non-trivial and unsolved problem especially in the asymmetric scenario where the query is unimodal while the item is multimodal. In this paper, we propose a novel model named SMAR, which stands for Semantic-enhanced Modality-Asymmetric Retrieval, to tackle the problem of modality fusion and alignment in this kind of asymmetric scenario. Extensive experimental results on an industrial dataset show that the proposed model outperforms baseline models significantly in retrieval accuracy. We have open sourced our industrial dataset for the sake of reproducibility and future research works.
title	Semantic-enhanced Modality-asymmetric Retrieval for Online E-commerce Search
topic	Information Retrieval
url	https://arxiv.org/abs/2506.20330