LREA: Low-Rank Efficient Attention on Modeling Long-Term User Behaviors for CTR Prediction
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910932218675200 |
|---|---|
| author | Song, Xin Li, Xiaochen Hu, Jinxin Wen, Hong Chen, Zulong Zhang, Yu Zeng, Xiaoyi Zhang, Jing |
| author_facet | Song, Xin Li, Xiaochen Hu, Jinxin Wen, Hong Chen, Zulong Zhang, Yu Zeng, Xiaoyi Zhang, Jing |
| contents | With the rapid growth of user historical behavior data, user interest modeling has become a prominent aspect in Click-Through Rate (CTR) prediction, focusing on learning user intent representations. However, this complexity poses computational challenges, requiring a balance between model performance and acceptable response times for online services. Traditional methods often utilize filtering techniques. These techniques can lead to the loss of significant information by prioritizing top K items based on item attributes or employing low-precision attention mechanisms. In this study, we introduce LREA, a novel attention mechanism that overcomes the limitations of existing approaches while ensuring computational efficiency. LREA leverages low-rank matrix decomposition to optimize runtime performance and incorporates a specially designed loss function to maintain attention capabilities while preserving information integrity. During the inference phase, matrix absorption and pre-storage strategies are employed to effectively meet runtime constraints. The results of extensive offline and online experiments demonstrate that our method outperforms state-of-the-art approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_02542 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LREA: Low-Rank Efficient Attention on Modeling Long-Term User Behaviors for CTR Prediction Song, Xin Li, Xiaochen Hu, Jinxin Wen, Hong Chen, Zulong Zhang, Yu Zeng, Xiaoyi Zhang, Jing Information Retrieval With the rapid growth of user historical behavior data, user interest modeling has become a prominent aspect in Click-Through Rate (CTR) prediction, focusing on learning user intent representations. However, this complexity poses computational challenges, requiring a balance between model performance and acceptable response times for online services. Traditional methods often utilize filtering techniques. These techniques can lead to the loss of significant information by prioritizing top K items based on item attributes or employing low-precision attention mechanisms. In this study, we introduce LREA, a novel attention mechanism that overcomes the limitations of existing approaches while ensuring computational efficiency. LREA leverages low-rank matrix decomposition to optimize runtime performance and incorporates a specially designed loss function to maintain attention capabilities while preserving information integrity. During the inference phase, matrix absorption and pre-storage strategies are employed to effectively meet runtime constraints. The results of extensive offline and online experiments demonstrate that our method outperforms state-of-the-art approaches. |
| title | LREA: Low-Rank Efficient Attention on Modeling Long-Term User Behaviors for CTR Prediction |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2503.02542 |