Towards Scalable Semantic Representation for Recommendation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913543544111104 |
|---|---|
| author | Zhang, Taolin Pan, Junwei Wang, Jinpeng Zha, Yaohua Dai, Tao Chen, Bin Luo, Ruisheng Deng, Xiaoxiang Wang, Yuan Yue, Ming Jiang, Jie Xia, Shu-Tao |
| author_facet | Zhang, Taolin Pan, Junwei Wang, Jinpeng Zha, Yaohua Dai, Tao Chen, Bin Luo, Ruisheng Deng, Xiaoxiang Wang, Yuan Yue, Ming Jiang, Jie Xia, Shu-Tao |
| contents | With recent advances in large language models (LLMs), there has been emerging numbers of research in developing Semantic IDs based on LLMs to enhance the performance of recommendation systems. However, the dimension of these embeddings needs to match that of the ID embedding in recommendation, which is usually much smaller than the original length. Such dimension compression results in inevitable losses in discriminability and dimension robustness of the LLM embeddings, which motivates us to scale up the semantic representation. In this paper, we propose Mixture-of-Codes, which first constructs multiple independent codebooks for LLM representation in the indexing stage, and then utilizes the Semantic Representation along with a fusion module for the downstream recommendation stage. Extensive analysis and experiments demonstrate that our method achieves superior discriminability and dimension robustness scalability, leading to the best scale-up performance in recommendations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_09560 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Scalable Semantic Representation for Recommendation Zhang, Taolin Pan, Junwei Wang, Jinpeng Zha, Yaohua Dai, Tao Chen, Bin Luo, Ruisheng Deng, Xiaoxiang Wang, Yuan Yue, Ming Jiang, Jie Xia, Shu-Tao Information Retrieval Machine Learning With recent advances in large language models (LLMs), there has been emerging numbers of research in developing Semantic IDs based on LLMs to enhance the performance of recommendation systems. However, the dimension of these embeddings needs to match that of the ID embedding in recommendation, which is usually much smaller than the original length. Such dimension compression results in inevitable losses in discriminability and dimension robustness of the LLM embeddings, which motivates us to scale up the semantic representation. In this paper, we propose Mixture-of-Codes, which first constructs multiple independent codebooks for LLM representation in the indexing stage, and then utilizes the Semantic Representation along with a fusion module for the downstream recommendation stage. Extensive analysis and experiments demonstrate that our method achieves superior discriminability and dimension robustness scalability, leading to the best scale-up performance in recommendations. |
| title | Towards Scalable Semantic Representation for Recommendation |
| topic | Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2410.09560 |