HistLLM: A Unified Framework for LLM-Based Multimodal Recommendation with User History Encoding and Compression
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916701239508992 |
|---|---|
| author | Zhang, Chen Hu, Bo Chen, Weidong Mao, Zhendong |
| author_facet | Zhang, Chen Hu, Bo Chen, Weidong Mao, Zhendong |
| contents | While large language models (LLMs) have proven effective in leveraging textual data for recommendations, their application to multimodal recommendation tasks remains relatively underexplored. Although LLMs can process multimodal information through projection functions that map visual features into their semantic space, recommendation tasks often require representing users' history interactions through lengthy prompts combining text and visual elements, which not only hampers training and inference efficiency but also makes it difficult for the model to accurately capture user preferences from complex and extended prompts, leading to reduced recommendation performance. To address this challenge, we introduce HistLLM, an innovative multimodal recommendation framework that integrates textual and visual features through a User History Encoding Module (UHEM), compressing multimodal user history interactions into a single token representation, effectively facilitating LLMs in processing user preferences. Extensive experiments demonstrate the effectiveness and efficiency of our proposed mechanism. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_10150 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HistLLM: A Unified Framework for LLM-Based Multimodal Recommendation with User History Encoding and Compression Zhang, Chen Hu, Bo Chen, Weidong Mao, Zhendong Information Retrieval Multimedia While large language models (LLMs) have proven effective in leveraging textual data for recommendations, their application to multimodal recommendation tasks remains relatively underexplored. Although LLMs can process multimodal information through projection functions that map visual features into their semantic space, recommendation tasks often require representing users' history interactions through lengthy prompts combining text and visual elements, which not only hampers training and inference efficiency but also makes it difficult for the model to accurately capture user preferences from complex and extended prompts, leading to reduced recommendation performance. To address this challenge, we introduce HistLLM, an innovative multimodal recommendation framework that integrates textual and visual features through a User History Encoding Module (UHEM), compressing multimodal user history interactions into a single token representation, effectively facilitating LLMs in processing user preferences. Extensive experiments demonstrate the effectiveness and efficiency of our proposed mechanism. |
| title | HistLLM: A Unified Framework for LLM-Based Multimodal Recommendation with User History Encoding and Compression |
| topic | Information Retrieval Multimedia |
| url | https://arxiv.org/abs/2504.10150 |