Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.01361 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929727925649408 |
|---|---|
| author | Veviurko, Grigorii Böhmer, Wendelin de Weerdt, Mathijs |
| author_facet | Veviurko, Grigorii Böhmer, Wendelin de Weerdt, Mathijs |
| contents | In reinforcement learning (RL), different reward functions can define the same optimal policy but result in drastically different learning performance. For some, the agent gets stuck with a suboptimal behavior, and for others, it solves the task efficiently. Choosing a good reward function is hence an extremely important yet challenging problem. In this paper, we explore an alternative approach for using rewards for learning. We introduce \textit{max-reward RL}, where an agent optimizes the maximum rather than the cumulative reward. Unlike earlier works, our approach works for deterministic and stochastic environments and can be easily combined with state-of-the-art RL algorithms. In the experiments, we study the performance of max-reward RL algorithms in two goal-reaching environments from Gymnasium-Robotics and demonstrate its benefits over standard RL. The code is available at https://github.com/veviurko/To-the-Max. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_01361 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | To the Max: Reinventing Reward in Reinforcement Learning Veviurko, Grigorii Böhmer, Wendelin de Weerdt, Mathijs Machine Learning In reinforcement learning (RL), different reward functions can define the same optimal policy but result in drastically different learning performance. For some, the agent gets stuck with a suboptimal behavior, and for others, it solves the task efficiently. Choosing a good reward function is hence an extremely important yet challenging problem. In this paper, we explore an alternative approach for using rewards for learning. We introduce \textit{max-reward RL}, where an agent optimizes the maximum rather than the cumulative reward. Unlike earlier works, our approach works for deterministic and stochastic environments and can be easily combined with state-of-the-art RL algorithms. In the experiments, we study the performance of max-reward RL algorithms in two goal-reaching environments from Gymnasium-Robotics and demonstrate its benefits over standard RL. The code is available at https://github.com/veviurko/To-the-Max. |
| title | To the Max: Reinventing Reward in Reinforcement Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.01361 |