PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911199788007424 |
|---|---|
| author | He, Ruining Heldt, Lukasz Hong, Lichan Keshavan, Raghunandan Mao, Shifan Mehta, Nikhil Su, Zhengyang Tsai, Alicia Wang, Yueqi Wang, Shao-Chuan Yi, Xinyang Baugher, Lexi Cakici, Baykal Chi, Ed Goodrow, Cristos Han, Ningren Ma, He Rosales, Romer Van Soest, Abby Tandon, Devansh Wu, Su-Lin Yang, Weilong Zheng, Yilin |
| author_facet | He, Ruining Heldt, Lukasz Hong, Lichan Keshavan, Raghunandan Mao, Shifan Mehta, Nikhil Su, Zhengyang Tsai, Alicia Wang, Yueqi Wang, Shao-Chuan Yi, Xinyang Baugher, Lexi Cakici, Baykal Chi, Ed Goodrow, Cristos Han, Ningren Ma, He Rosales, Romer Van Soest, Abby Tandon, Devansh Wu, Su-Lin Yang, Weilong Zheng, Yilin |
| contents | Large Language Models (LLMs) pose a new paradigm of modeling and computation for information tasks. Recommendation systems are a critical application domain poised to benefit significantly from the sequence modeling capabilities and world knowledge inherent in these large models. In this paper, we introduce PLUM, a framework designed to adapt pre-trained LLMs for industry-scale recommendation tasks. PLUM consists of item tokenization using Semantic IDs, continued pre-training (CPT) on domain-specific data, and task-specific fine-tuning for recommendation objectives. For fine-tuning, we focus particularly on generative retrieval, where the model is directly trained to generate Semantic IDs of recommended items based on user context. We conduct comprehensive experiments on large-scale internal video recommendation datasets. Our results demonstrate that PLUM achieves substantial improvements for retrieval compared to a heavily-optimized production model built with large embedding tables. We also present a scaling study for the model's retrieval performance, our learnings about CPT, a few enhancements to Semantic IDs, along with an overview of the training and inference methods that enable launching this framework to billions of users in YouTube. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_07784 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations He, Ruining Heldt, Lukasz Hong, Lichan Keshavan, Raghunandan Mao, Shifan Mehta, Nikhil Su, Zhengyang Tsai, Alicia Wang, Yueqi Wang, Shao-Chuan Yi, Xinyang Baugher, Lexi Cakici, Baykal Chi, Ed Goodrow, Cristos Han, Ningren Ma, He Rosales, Romer Van Soest, Abby Tandon, Devansh Wu, Su-Lin Yang, Weilong Zheng, Yilin Information Retrieval Machine Learning Large Language Models (LLMs) pose a new paradigm of modeling and computation for information tasks. Recommendation systems are a critical application domain poised to benefit significantly from the sequence modeling capabilities and world knowledge inherent in these large models. In this paper, we introduce PLUM, a framework designed to adapt pre-trained LLMs for industry-scale recommendation tasks. PLUM consists of item tokenization using Semantic IDs, continued pre-training (CPT) on domain-specific data, and task-specific fine-tuning for recommendation objectives. For fine-tuning, we focus particularly on generative retrieval, where the model is directly trained to generate Semantic IDs of recommended items based on user context. We conduct comprehensive experiments on large-scale internal video recommendation datasets. Our results demonstrate that PLUM achieves substantial improvements for retrieval compared to a heavily-optimized production model built with large embedding tables. We also present a scaling study for the model's retrieval performance, our learnings about CPT, a few enhancements to Semantic IDs, along with an overview of the training and inference methods that enable launching this framework to billions of users in YouTube. |
| title | PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations |
| topic | Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2510.07784 |