RAT: Retrieval-Augmented Transformer for Click-Through Rate Prediction
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916194526691328 |
|---|---|
| author | Li, Yushen Wang, Jinpeng Dai, Tao Zhu, Jieming Yuan, Jun Zhang, Rui Xia, Shu-Tao |
| author_facet | Li, Yushen Wang, Jinpeng Dai, Tao Zhu, Jieming Yuan, Jun Zhang, Rui Xia, Shu-Tao |
| contents | Predicting click-through rates (CTR) is a fundamental task for Web applications, where a key issue is to devise effective models for feature interactions. Current methodologies predominantly concentrate on modeling feature interactions within an individual sample, while overlooking the potential cross-sample relationships that can serve as a reference context to enhance the prediction. To make up for such deficiency, this paper develops a Retrieval-Augmented Transformer (RAT), aiming to acquire fine-grained feature interactions within and across samples. By retrieving similar samples, we construct augmented input for each target sample. We then build Transformer layers with cascaded attention to capture both intra- and cross-sample feature interactions, facilitating comprehensive reasoning for improved CTR prediction while retaining efficiency. Extensive experiments on real-world datasets substantiate the effectiveness of RAT and suggest its advantage in long-tail scenarios. The code has been open-sourced at \url{https://github.com/YushenLi807/WWW24-RAT}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_02249 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RAT: Retrieval-Augmented Transformer for Click-Through Rate Prediction Li, Yushen Wang, Jinpeng Dai, Tao Zhu, Jieming Yuan, Jun Zhang, Rui Xia, Shu-Tao Information Retrieval Artificial Intelligence Machine Learning Social and Information Networks Predicting click-through rates (CTR) is a fundamental task for Web applications, where a key issue is to devise effective models for feature interactions. Current methodologies predominantly concentrate on modeling feature interactions within an individual sample, while overlooking the potential cross-sample relationships that can serve as a reference context to enhance the prediction. To make up for such deficiency, this paper develops a Retrieval-Augmented Transformer (RAT), aiming to acquire fine-grained feature interactions within and across samples. By retrieving similar samples, we construct augmented input for each target sample. We then build Transformer layers with cascaded attention to capture both intra- and cross-sample feature interactions, facilitating comprehensive reasoning for improved CTR prediction while retaining efficiency. Extensive experiments on real-world datasets substantiate the effectiveness of RAT and suggest its advantage in long-tail scenarios. The code has been open-sourced at \url{https://github.com/YushenLi807/WWW24-RAT}. |
| title | RAT: Retrieval-Augmented Transformer for Click-Through Rate Prediction |
| topic | Information Retrieval Artificial Intelligence Machine Learning Social and Information Networks |
| url | https://arxiv.org/abs/2404.02249 |