Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.07755 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915102102388736 |
|---|---|
| author | Rose, Evelyn White, Devin Wu, Mingkang Lawhern, Vernon Waytowich, Nicholas R. Cao, Yongcan |
| author_facet | Rose, Evelyn White, Devin Wu, Mingkang Lawhern, Vernon Waytowich, Nicholas R. Cao, Yongcan |
| contents | This paper explores multiple optimization methods to improve the performance of rating-based reinforcement learning (RbRL). RbRL, a method based on the idea of human ratings, has been developed to infer reward functions in reward-free environments for the subsequent policy learning via standard reinforcement learning, which requires the availability of reward functions. Specifically, RbRL minimizes the cross entropy loss that quantifies the differences between human ratings and estimated ratings derived from the inferred reward. Hence, a low loss means a high degree of consistency between human ratings and estimated ratings. Despite its simple form, RbRL has various hyperparameters and can be sensitive to various factors. Therefore, it is critical to provide comprehensive experiments to understand the impact of various hyperparameters on the performance of RbRL. This paper is a work in progress, providing users some general guidelines on how to select hyperparameters in RbRL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_07755 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Performance Optimization of Ratings-Based Reinforcement Learning Rose, Evelyn White, Devin Wu, Mingkang Lawhern, Vernon Waytowich, Nicholas R. Cao, Yongcan Machine Learning Artificial Intelligence This paper explores multiple optimization methods to improve the performance of rating-based reinforcement learning (RbRL). RbRL, a method based on the idea of human ratings, has been developed to infer reward functions in reward-free environments for the subsequent policy learning via standard reinforcement learning, which requires the availability of reward functions. Specifically, RbRL minimizes the cross entropy loss that quantifies the differences between human ratings and estimated ratings derived from the inferred reward. Hence, a low loss means a high degree of consistency between human ratings and estimated ratings. Despite its simple form, RbRL has various hyperparameters and can be sensitive to various factors. Therefore, it is critical to provide comprehensive experiments to understand the impact of various hyperparameters on the performance of RbRL. This paper is a work in progress, providing users some general guidelines on how to select hyperparameters in RbRL. |
| title | Performance Optimization of Ratings-Based Reinforcement Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2501.07755 |