Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systems
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912472690065408 |
|---|---|
| author | Liu, Langming Wang, Wanyu Zhang, Chi Li, Bo Yin, Hongzhi Wei, Xuetao Su, Wenbo Zheng, Bo Zhao, Xiangyu |
| author_facet | Liu, Langming Wang, Wanyu Zhang, Chi Li, Bo Yin, Hongzhi Wei, Xuetao Su, Wenbo Zheng, Bo Zhao, Xiangyu |
| contents | Online advertising in recommendation platforms has gained significant attention, with a predominant focus on channel recommendation and budget allocation strategies. However, current offline reinforcement learning (RL) methods face substantial challenges when applied to sparse advertising scenarios, primarily due to severe overestimation, distributional shifts, and overlooking budget constraints. To address these issues, we propose MTORL, a novel multi-task offline RL model that targets two key objectives. First, we establish a Markov Decision Process (MDP) framework specific to the nuances of advertising. Then, we develop a causal state encoder to capture dynamic user interests and temporal dependencies, facilitating offline RL through conditional sequence modeling. Causal attention mechanisms are introduced to enhance user sequence representations by identifying correlations among causal states. We employ multi-task learning to decode actions and rewards, simultaneously addressing channel recommendation and budget allocation. Notably, our framework includes an automated system for integrating these tasks into online advertising. Extensive experiments on offline and online environments demonstrate MTORL's superiority over state-of-the-art methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_23090 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systems Liu, Langming Wang, Wanyu Zhang, Chi Li, Bo Yin, Hongzhi Wei, Xuetao Su, Wenbo Zheng, Bo Zhao, Xiangyu Information Retrieval Machine Learning Online advertising in recommendation platforms has gained significant attention, with a predominant focus on channel recommendation and budget allocation strategies. However, current offline reinforcement learning (RL) methods face substantial challenges when applied to sparse advertising scenarios, primarily due to severe overestimation, distributional shifts, and overlooking budget constraints. To address these issues, we propose MTORL, a novel multi-task offline RL model that targets two key objectives. First, we establish a Markov Decision Process (MDP) framework specific to the nuances of advertising. Then, we develop a causal state encoder to capture dynamic user interests and temporal dependencies, facilitating offline RL through conditional sequence modeling. Causal attention mechanisms are introduced to enhance user sequence representations by identifying correlations among causal states. We employ multi-task learning to decode actions and rewards, simultaneously addressing channel recommendation and budget allocation. Notably, our framework includes an automated system for integrating these tasks into online advertising. Extensive experiments on offline and online environments demonstrate MTORL's superiority over state-of-the-art methods. |
| title | Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systems |
| topic | Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2506.23090 |