A General Framework for Multimodal LLM-Based Multimedia Understanding in Large-Scale Recommendation Systems
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909030061965312 |
|---|---|
| author | Zhu, Yiming Liu, Xu Xu, Ziyun Wu, Zheng Zhang, Joena Chen, Sirius Hua, Chenheli Yao, Silvester Que, Qichao Shi, Wentao Pan, Junfeng Zhu, Linhong |
| author_facet | Zhu, Yiming Liu, Xu Xu, Ziyun Wu, Zheng Zhang, Joena Chen, Sirius Hua, Chenheli Yao, Silvester Que, Qichao Shi, Wentao Pan, Junfeng Zhu, Linhong |
| contents | Conventional recommendation systems frequently fail to fully exploit the high-dimensional semantic signals inherent in multimedia content, thereby limiting the fidelity of user preference modeling. While Multimodal Large Language Models (MM-LLMs) offer robust mechanisms for interpreting such complex data, their integration into latency-constrained, industrial-scale architectures remains a significant challenge. To address this, we propose a generalized framework for MM-LLM-driven multimedia understanding. Our methodology employs a tripartite architecture encompassing content interpretation, representation extraction, and systematic pipeline integration, instantiated via a LLaMA2-based model that generates descriptive captions subsequently ingested as tokenized categorical features. Empirical evaluation demonstrates the efficacy of this approach, yielding a $0.35\%$ increase in offline AUC and a $0.02\%$ improvement in online metrics at scale, substantiating the practical viability of leveraging MM-LLMs to enhance large-scale recommendation performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_09338 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A General Framework for Multimodal LLM-Based Multimedia Understanding in Large-Scale Recommendation Systems Zhu, Yiming Liu, Xu Xu, Ziyun Wu, Zheng Zhang, Joena Chen, Sirius Hua, Chenheli Yao, Silvester Que, Qichao Shi, Wentao Pan, Junfeng Zhu, Linhong Information Retrieval Conventional recommendation systems frequently fail to fully exploit the high-dimensional semantic signals inherent in multimedia content, thereby limiting the fidelity of user preference modeling. While Multimodal Large Language Models (MM-LLMs) offer robust mechanisms for interpreting such complex data, their integration into latency-constrained, industrial-scale architectures remains a significant challenge. To address this, we propose a generalized framework for MM-LLM-driven multimedia understanding. Our methodology employs a tripartite architecture encompassing content interpretation, representation extraction, and systematic pipeline integration, instantiated via a LLaMA2-based model that generates descriptive captions subsequently ingested as tokenized categorical features. Empirical evaluation demonstrates the efficacy of this approach, yielding a $0.35\%$ increase in offline AUC and a $0.02\%$ improvement in online metrics at scale, substantiating the practical viability of leveraging MM-LLMs to enhance large-scale recommendation performance. |
| title | A General Framework for Multimodal LLM-Based Multimedia Understanding in Large-Scale Recommendation Systems |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2605.09338 |