LLMInit: A Free Lunch from Large Language Models for Selective Initialization of Recommendation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912719821602816 |
|---|---|
| author | Zhang, Weizhi Yang, Liangwei Yang, Wooseong Zou, Henry Peng Liu, Yuqing Xu, Ke Medya, Sourav Yu, Philip S. |
| author_facet | Zhang, Weizhi Yang, Liangwei Yang, Wooseong Zou, Henry Peng Liu, Yuqing Xu, Ke Medya, Sourav Yu, Philip S. |
| contents | Collaborative filtering (CF) is widely adopted in industrial recommender systems (RecSys) for modeling user-item interactions across numerous applications, but often struggles with cold-start and data-sparse scenarios. Recent advancements in pre-trained large language models (LLMs) with rich semantic knowledge, offer promising solutions to these challenges. However, deploying LLMs at scale is hindered by their significant computational demands and latency. In this paper, we propose a novel and scalable LLM-RecSys framework, LLMInit, designed to integrate pretrained LLM embeddings into CF models through selective initialization strategies. Specifically, we identify the embedding collapse issue observed when CF models scale and match the large embedding sizes in LLMs and avoid the problem by introducing efficient sampling methods, including, random, uniform, and variance-based selections. Comprehensive experiments conducted on multiple real-world datasets demonstrate that LLMInit significantly improves recommendation performance while maintaining low computational costs, offering a practical and scalable solution for industrial applications. To facilitate industry adoption and promote future research, we provide open-source access to our implementation at https://github.com/DavidZWZ/LLMInit. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_01814 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLMInit: A Free Lunch from Large Language Models for Selective Initialization of Recommendation Zhang, Weizhi Yang, Liangwei Yang, Wooseong Zou, Henry Peng Liu, Yuqing Xu, Ke Medya, Sourav Yu, Philip S. Information Retrieval Artificial Intelligence Computation and Language Machine Learning Collaborative filtering (CF) is widely adopted in industrial recommender systems (RecSys) for modeling user-item interactions across numerous applications, but often struggles with cold-start and data-sparse scenarios. Recent advancements in pre-trained large language models (LLMs) with rich semantic knowledge, offer promising solutions to these challenges. However, deploying LLMs at scale is hindered by their significant computational demands and latency. In this paper, we propose a novel and scalable LLM-RecSys framework, LLMInit, designed to integrate pretrained LLM embeddings into CF models through selective initialization strategies. Specifically, we identify the embedding collapse issue observed when CF models scale and match the large embedding sizes in LLMs and avoid the problem by introducing efficient sampling methods, including, random, uniform, and variance-based selections. Comprehensive experiments conducted on multiple real-world datasets demonstrate that LLMInit significantly improves recommendation performance while maintaining low computational costs, offering a practical and scalable solution for industrial applications. To facilitate industry adoption and promote future research, we provide open-source access to our implementation at https://github.com/DavidZWZ/LLMInit. |
| title | LLMInit: A Free Lunch from Large Language Models for Selective Initialization of Recommendation |
| topic | Information Retrieval Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2503.01814 |