Learning Decomposed Contextual Token Representations from Pretrained and Collaborative Signals for Generative Recommendation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913081645334528 |
|---|---|
| author | Liu, Yifan Liu, Yaokun Li, Zelin Yue, Zhenrui Lee, Gyuseok Yao, Ruichen Zhang, Yang Wang, Dong |
| author_facet | Liu, Yifan Liu, Yaokun Li, Zelin Yue, Zhenrui Lee, Gyuseok Yao, Ruichen Zhang, Yang Wang, Dong |
| contents | Recent advances in generative recommenders adopt a two-stage paradigm: items are first tokenized into semantic IDs using a pretrained tokenizer, and then large language models (LLMs) are trained to generate the next item via sequence-to-sequence modeling. However, these two stages are optimized for different objectives: semantic reconstruction during tokenizer pretraining versus user interaction modeling during recommender training. This objective misalignment leads to two key limitations: (i) suboptimal static tokenization, where fixed token assignments fail to reflect diverse usage contexts; and (ii) discarded pretrained semantics, where pretrained knowledge - typically from language model embeddings - is overwritten during recommender training on user interactions. To address these limitations, we propose to learn $\underline{DE}$composed $\underline{CO}$ntextual Token $\underline{R}$epresentations (DECOR), a unified framework that preserves pretrained semantics while enhancing the adaptability of token embeddings. DECOR introduces contextualized token composition to refine token embeddings based on user interaction context, and decomposed embedding fusion that integrates pretrained codebook embeddings with newly learned collaborative embeddings. Experiments on three real-world datasets demonstrate that DECOR consistently outperforms state-of-the-art baselines in recommendation performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_10468 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning Decomposed Contextual Token Representations from Pretrained and Collaborative Signals for Generative Recommendation Liu, Yifan Liu, Yaokun Li, Zelin Yue, Zhenrui Lee, Gyuseok Yao, Ruichen Zhang, Yang Wang, Dong Information Retrieval Artificial Intelligence Computation and Language cs.IR Recent advances in generative recommenders adopt a two-stage paradigm: items are first tokenized into semantic IDs using a pretrained tokenizer, and then large language models (LLMs) are trained to generate the next item via sequence-to-sequence modeling. However, these two stages are optimized for different objectives: semantic reconstruction during tokenizer pretraining versus user interaction modeling during recommender training. This objective misalignment leads to two key limitations: (i) suboptimal static tokenization, where fixed token assignments fail to reflect diverse usage contexts; and (ii) discarded pretrained semantics, where pretrained knowledge - typically from language model embeddings - is overwritten during recommender training on user interactions. To address these limitations, we propose to learn $\underline{DE}$composed $\underline{CO}$ntextual Token $\underline{R}$epresentations (DECOR), a unified framework that preserves pretrained semantics while enhancing the adaptability of token embeddings. DECOR introduces contextualized token composition to refine token embeddings based on user interaction context, and decomposed embedding fusion that integrates pretrained codebook embeddings with newly learned collaborative embeddings. Experiments on three real-world datasets demonstrate that DECOR consistently outperforms state-of-the-art baselines in recommendation performance. |
| title | Learning Decomposed Contextual Token Representations from Pretrained and Collaborative Signals for Generative Recommendation |
| topic | Information Retrieval Artificial Intelligence Computation and Language cs.IR |
| url | https://arxiv.org/abs/2509.10468 |