Bidirectional Knowledge Distillation for Enhancing Sequential Recommendation with Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918032185491456 |
|---|---|
| author | Wu, Jiongran Liu, Jiahao Li, Dongsheng Zhang, Guangping Han, Mingzhe Gu, Hansu Zhang, Peng Shang, Li Lu, Tun Gu, Ning |
| author_facet | Wu, Jiongran Liu, Jiahao Li, Dongsheng Zhang, Guangping Han, Mingzhe Gu, Hansu Zhang, Peng Shang, Li Lu, Tun Gu, Ning |
| contents | Large language models (LLMs) have demonstrated exceptional performance in understanding and generating semantic patterns, making them promising candidates for sequential recommendation tasks. However, when combined with conventional recommendation models (CRMs), LLMs often face challenges related to high inference costs and static knowledge transfer methods. In this paper, we propose a novel mutual distillation framework, LLMD4Rec, that fosters dynamic and bidirectional knowledge exchange between LLM-centric and CRM-based recommendation systems. Unlike traditional unidirectional distillation methods, LLMD4Rec enables iterative optimization by alternately refining both models, enhancing the semantic understanding of CRMs and enriching LLMs with collaborative signals from user-item interactions. By leveraging sample-wise adaptive weighting and aligning output distributions, our approach eliminates the need for additional parameters while ensuring effective knowledge transfer. Extensive experiments on real-world datasets demonstrate that LLMD4Rec significantly improves recommendation accuracy across multiple benchmarks without increasing inference costs. This method provides a scalable and efficient solution for combining the strengths of both LLMs and CRMs in sequential recommendation systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18120 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bidirectional Knowledge Distillation for Enhancing Sequential Recommendation with Large Language Models Wu, Jiongran Liu, Jiahao Li, Dongsheng Zhang, Guangping Han, Mingzhe Gu, Hansu Zhang, Peng Shang, Li Lu, Tun Gu, Ning Information Retrieval Artificial Intelligence Large language models (LLMs) have demonstrated exceptional performance in understanding and generating semantic patterns, making them promising candidates for sequential recommendation tasks. However, when combined with conventional recommendation models (CRMs), LLMs often face challenges related to high inference costs and static knowledge transfer methods. In this paper, we propose a novel mutual distillation framework, LLMD4Rec, that fosters dynamic and bidirectional knowledge exchange between LLM-centric and CRM-based recommendation systems. Unlike traditional unidirectional distillation methods, LLMD4Rec enables iterative optimization by alternately refining both models, enhancing the semantic understanding of CRMs and enriching LLMs with collaborative signals from user-item interactions. By leveraging sample-wise adaptive weighting and aligning output distributions, our approach eliminates the need for additional parameters while ensuring effective knowledge transfer. Extensive experiments on real-world datasets demonstrate that LLMD4Rec significantly improves recommendation accuracy across multiple benchmarks without increasing inference costs. This method provides a scalable and efficient solution for combining the strengths of both LLMs and CRMs in sequential recommendation systems. |
| title | Bidirectional Knowledge Distillation for Enhancing Sequential Recommendation with Large Language Models |
| topic | Information Retrieval Artificial Intelligence |
| url | https://arxiv.org/abs/2505.18120 |