Multimodal Generative Recommendation for Fusing Semantic and Collaborative Signals
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915771696807936 |
|---|---|
| author | Vandenhirtz, Moritz Hassani, Kaveh Ghasemlou, Shervin Shao, Shuai Eghbalzadeh, Hamid Peng, Fuchun Liu, Jun Iuzzolino, Michael Louis |
| author_facet | Vandenhirtz, Moritz Hassani, Kaveh Ghasemlou, Shervin Shao, Shuai Eghbalzadeh, Hamid Peng, Fuchun Liu, Jun Iuzzolino, Michael Louis |
| contents | Sequential recommender systems rank relevant items by modeling a user's interaction history and computing the inner product between the resulting user representation and stored item embeddings. To avoid the significant memory overhead of storing large item sets, the generative recommendation paradigm instead models each item as a series of discrete semantic codes. Here, the next item is predicted by an autoregressive model that generates the code sequence corresponding to the predicted item. However, despite promising ranking capabilities on small datasets, these methods have yet to surpass traditional sequential recommenders on large item sets, limiting their adoption in the very scenarios they were designed to address. To resolve this, we propose MSCGRec, a Multimodal Semantic and Collaborative Generative Recommender. MSCGRec incorporates multiple semantic modalities and introduces a novel self-supervised quantization learning approach for images based on the DINO framework. Additionally, MSCGRec fuses collaborative and semantic signals by extracting collaborative features from sequential recommenders and treating them as a separate modality. Finally, we propose constrained sequence learning that restricts the large output space during training to the set of permissible tokens. We empirically demonstrate on three large real-world datasets that MSCGRec outperforms both sequential and generative recommendation baselines and provide an extensive ablation study to validate the impact of each component. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_03713 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Multimodal Generative Recommendation for Fusing Semantic and Collaborative Signals Vandenhirtz, Moritz Hassani, Kaveh Ghasemlou, Shervin Shao, Shuai Eghbalzadeh, Hamid Peng, Fuchun Liu, Jun Iuzzolino, Michael Louis Information Retrieval Sequential recommender systems rank relevant items by modeling a user's interaction history and computing the inner product between the resulting user representation and stored item embeddings. To avoid the significant memory overhead of storing large item sets, the generative recommendation paradigm instead models each item as a series of discrete semantic codes. Here, the next item is predicted by an autoregressive model that generates the code sequence corresponding to the predicted item. However, despite promising ranking capabilities on small datasets, these methods have yet to surpass traditional sequential recommenders on large item sets, limiting their adoption in the very scenarios they were designed to address. To resolve this, we propose MSCGRec, a Multimodal Semantic and Collaborative Generative Recommender. MSCGRec incorporates multiple semantic modalities and introduces a novel self-supervised quantization learning approach for images based on the DINO framework. Additionally, MSCGRec fuses collaborative and semantic signals by extracting collaborative features from sequential recommenders and treating them as a separate modality. Finally, we propose constrained sequence learning that restricts the large output space during training to the set of permissible tokens. We empirically demonstrate on three large real-world datasets that MSCGRec outperforms both sequential and generative recommendation baselines and provide an extensive ablation study to validate the impact of each component. |
| title | Multimodal Generative Recommendation for Fusing Semantic and Collaborative Signals |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2602.03713 |