LEMUR: Large scale End-to-end MUltimodal Recommendation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908658993987584 |
|---|---|
| author | Han, Xintian Chen, Honggang Lin, Quan Gao, Jingyue Ren, Xiangyuan Zhu, Lifei Ye, Zhisheng Wu, Shikang Xie, XiongHang Gan, Xiaochu Wei, Bingzheng Xu, Peng Wang, Zhe Zheng, Yuchao Lin, Jingjian Wu, Di Ge, Junfeng |
| author_facet | Han, Xintian Chen, Honggang Lin, Quan Gao, Jingyue Ren, Xiangyuan Zhu, Lifei Ye, Zhisheng Wu, Shikang Xie, XiongHang Gan, Xiaochu Wei, Bingzheng Xu, Peng Wang, Zhe Zheng, Yuchao Lin, Jingjian Wu, Di Ge, Junfeng |
| contents | Traditional ID-based recommender systems often struggle with cold-start and generalization challenges. Multimodal recommendation systems, which leverage textual and visual data, offer a promising solution to mitigate these issues. However, existing industrial approaches typically adopt a two-stage training paradigm: first pretraining a multimodal model, then applying its frozen representations to train the recommendation model. This decoupled framework suffers from misalignment between multimodal learning and recommendation objectives, as well as an inability to adapt dynamically to new data. To address these limitations, we propose LEMUR, the first large-scale multimodal recommender system trained end-to-end from raw data. By jointly optimizing both the multimodal and recommendation components, LEMUR ensures tighter alignment with downstream objectives while enabling real-time parameter updates. Constructing multimodal sequential representations from user history often entails prohibitively high computational costs. To alleviate this bottleneck, we propose a novel memory bank mechanism that incrementally accumulates historical multimodal representations throughout the training process. After one month of deployment in Douyin Search, LEMUR has led to a 0.843% reduction in query change rate decay and a 0.81% improvement in QAUC. Additionally, LEMUR has shown significant gains across key offline metrics for Douyin Advertisement. Our results validate the superiority of end-to-end multimodal recommendation in real-world industrial scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_10962 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LEMUR: Large scale End-to-end MUltimodal Recommendation Han, Xintian Chen, Honggang Lin, Quan Gao, Jingyue Ren, Xiangyuan Zhu, Lifei Ye, Zhisheng Wu, Shikang Xie, XiongHang Gan, Xiaochu Wei, Bingzheng Xu, Peng Wang, Zhe Zheng, Yuchao Lin, Jingjian Wu, Di Ge, Junfeng Information Retrieval Traditional ID-based recommender systems often struggle with cold-start and generalization challenges. Multimodal recommendation systems, which leverage textual and visual data, offer a promising solution to mitigate these issues. However, existing industrial approaches typically adopt a two-stage training paradigm: first pretraining a multimodal model, then applying its frozen representations to train the recommendation model. This decoupled framework suffers from misalignment between multimodal learning and recommendation objectives, as well as an inability to adapt dynamically to new data. To address these limitations, we propose LEMUR, the first large-scale multimodal recommender system trained end-to-end from raw data. By jointly optimizing both the multimodal and recommendation components, LEMUR ensures tighter alignment with downstream objectives while enabling real-time parameter updates. Constructing multimodal sequential representations from user history often entails prohibitively high computational costs. To alleviate this bottleneck, we propose a novel memory bank mechanism that incrementally accumulates historical multimodal representations throughout the training process. After one month of deployment in Douyin Search, LEMUR has led to a 0.843% reduction in query change rate decay and a 0.81% improvement in QAUC. Additionally, LEMUR has shown significant gains across key offline metrics for Douyin Advertisement. Our results validate the superiority of end-to-end multimodal recommendation in real-world industrial scenarios. |
| title | LEMUR: Large scale End-to-end MUltimodal Recommendation |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2511.10962 |