The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910772341243904 |
|---|---|
| author | Zhou, Huixue Gu, Hengrui Liu, Xi Zhou, Kaixiong Liang, Mingfu Xiao, Yongkang Govindan, Srinivas Chawla, Piyush Yang, Jiyan Meng, Xiangfei Li, Huayu Zhang, Buyun Luo, Liang Chen, Wen-Yen Han, Yiping Long, Bo Zhang, Rui Chen, Tianlong |
| author_facet | Zhou, Huixue Gu, Hengrui Liu, Xi Zhou, Kaixiong Liang, Mingfu Xiao, Yongkang Govindan, Srinivas Chawla, Piyush Yang, Jiyan Meng, Xiangfei Li, Huayu Zhang, Buyun Luo, Liang Chen, Wen-Yen Han, Yiping Long, Bo Zhang, Rui Chen, Tianlong |
| contents | The deployment of Large Language Models (LLMs) in recommender systems for predicting Click-Through Rates (CTR) necessitates a delicate balance between computational efficiency and predictive accuracy. This paper presents an optimization framework that combines Retrieval-Augmented Generation (RAG) with an innovative multi-head early exit architecture to concurrently enhance both aspects. By integrating Graph Convolutional Networks (GCNs) as efficient retrieval mechanisms, we are able to significantly reduce data retrieval times while maintaining high model performance. The early exit strategy employed allows for dynamic termination of model inference, utilizing real-time predictive confidence assessments across multiple heads. This not only quickens the responsiveness of LLMs but also upholds or improves their accuracy, making it ideal for real-time application scenarios. Our experiments demonstrate how this architecture effectively decreases computation time without sacrificing the accuracy needed for reliable recommendation delivery, establishing a new standard for efficient, real-time LLM deployment in commercial systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_02173 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit Zhou, Huixue Gu, Hengrui Liu, Xi Zhou, Kaixiong Liang, Mingfu Xiao, Yongkang Govindan, Srinivas Chawla, Piyush Yang, Jiyan Meng, Xiangfei Li, Huayu Zhang, Buyun Luo, Liang Chen, Wen-Yen Han, Yiping Long, Bo Zhang, Rui Chen, Tianlong Information Retrieval Machine Learning The deployment of Large Language Models (LLMs) in recommender systems for predicting Click-Through Rates (CTR) necessitates a delicate balance between computational efficiency and predictive accuracy. This paper presents an optimization framework that combines Retrieval-Augmented Generation (RAG) with an innovative multi-head early exit architecture to concurrently enhance both aspects. By integrating Graph Convolutional Networks (GCNs) as efficient retrieval mechanisms, we are able to significantly reduce data retrieval times while maintaining high model performance. The early exit strategy employed allows for dynamic termination of model inference, utilizing real-time predictive confidence assessments across multiple heads. This not only quickens the responsiveness of LLMs but also upholds or improves their accuracy, making it ideal for real-time application scenarios. Our experiments demonstrate how this architecture effectively decreases computation time without sacrificing the accuracy needed for reliable recommendation delivery, establishing a new standard for efficient, real-time LLM deployment in commercial systems. |
| title | The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit |
| topic | Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2501.02173 |