Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.00383 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908681454485504 |
|---|---|
| author | Su, Jianhai Luo, Jinzhu Zhang, Qi |
| author_facet | Su, Jianhai Luo, Jinzhu Zhang, Qi |
| contents | We take the novel perspective of incorporating offline RL algorithms as subroutines of tabula rasa online RL. This is feasible because an online learning agent can repurpose its historical interactions as offline dataset. We formalize this idea into a framework that accommodates several variants of offline RL incorporation such as final policy recommendation and online fine-tuning. We further introduce convenient techniques to improve its effectiveness in enhancing online learning efficiency. Our extensive and systematic empirical analyses show that 1) the effectiveness of the proposed framework depends strongly on the nature of the task, 2) our proposed techniques greatly enhance its effectiveness, and 3) existing online fine-tuning methods are overall ineffective, calling for more research therein. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_00383 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An Empirical Study on the Effectiveness of Incorporating Offline RL As Online RL Subroutines Su, Jianhai Luo, Jinzhu Zhang, Qi Machine Learning Artificial Intelligence We take the novel perspective of incorporating offline RL algorithms as subroutines of tabula rasa online RL. This is feasible because an online learning agent can repurpose its historical interactions as offline dataset. We formalize this idea into a framework that accommodates several variants of offline RL incorporation such as final policy recommendation and online fine-tuning. We further introduce convenient techniques to improve its effectiveness in enhancing online learning efficiency. Our extensive and systematic empirical analyses show that 1) the effectiveness of the proposed framework depends strongly on the nature of the task, 2) our proposed techniques greatly enhance its effectiveness, and 3) existing online fine-tuning methods are overall ineffective, calling for more research therein. |
| title | An Empirical Study on the Effectiveness of Incorporating Offline RL As Online RL Subroutines |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2512.00383 |