Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908989445373952 |
|---|---|
| author | Severin, Nikita Kartushov, Danil Urzhumov, Vladislav Kulikov, Vladislav Konovalova, Oksana Grishanov, Alexey Klenitskiy, Anton Fatkulin, Artem Vasilev, Alexey Savchenko, Andrey Makarov, Ilya |
| author_facet | Severin, Nikita Kartushov, Danil Urzhumov, Vladislav Kulikov, Vladislav Konovalova, Oksana Grishanov, Alexey Klenitskiy, Anton Fatkulin, Artem Vasilev, Alexey Savchenko, Andrey Makarov, Ilya |
| contents | Sequential recommender systems have achieved significant success in modeling temporal user behavior but remain limited in capturing rich user semantics beyond interaction patterns. Large Language Models (LLMs) present opportunities to enhance user understanding with their reasoning capabilities, yet existing integration approaches create prohibitive inference costs in real time. To address these limitations, we present a novel knowledge distillation method that utilizes textual user profile generated by pre-trained LLMs into sequential recommenders without requiring LLM inference at serving time. The resulting approach maintains the inference efficiency of traditional sequential models while requiring neither architectural modifications nor LLM fine-tuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_21536 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation Severin, Nikita Kartushov, Danil Urzhumov, Vladislav Kulikov, Vladislav Konovalova, Oksana Grishanov, Alexey Klenitskiy, Anton Fatkulin, Artem Vasilev, Alexey Savchenko, Andrey Makarov, Ilya Information Retrieval Artificial Intelligence H.3.3; I.2.7 Sequential recommender systems have achieved significant success in modeling temporal user behavior but remain limited in capturing rich user semantics beyond interaction patterns. Large Language Models (LLMs) present opportunities to enhance user understanding with their reasoning capabilities, yet existing integration approaches create prohibitive inference costs in real time. To address these limitations, we present a novel knowledge distillation method that utilizes textual user profile generated by pre-trained LLMs into sequential recommenders without requiring LLM inference at serving time. The resulting approach maintains the inference efficiency of traditional sequential models while requiring neither architectural modifications nor LLM fine-tuning. |
| title | Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation |
| topic | Information Retrieval Artificial Intelligence H.3.3; I.2.7 |
| url | https://arxiv.org/abs/2604.21536 |