Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.19142 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916338135465984 |
|---|---|
| author | Cristea-Platon, Tudor Mazoure, Bogdan Susskind, Josh Talbott, Walter |
| author_facet | Cristea-Platon, Tudor Mazoure, Bogdan Susskind, Josh Talbott, Walter |
| contents | Reinforcement learning practitioners often avoid hierarchical policies, especially in image-based observation spaces. Typically, the single-task performance improvement over flat-policy counterparts does not justify the additional complexity associated with implementing a hierarchy. However, by introducing multiple decision-making levels, hierarchical policies can compose lower-level policies to more effectively generalize between tasks, highlighting the need for multi-task evaluations. We analyze the benefits of hierarchy through simulated multi-task robotic control experiments from pixels. Our results show that hierarchical policies trained with task conditioning can (1) increase performance on training tasks, (2) lead to improved reward and state-space generalizations in similar tasks, and (3) decrease the complexity of fine tuning required to solve novel tasks. Thus, we believe that hierarchical policies should be considered when building reinforcement learning architectures capable of generalizing between tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_19142 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On the benefits of pixel-based hierarchical policies for task generalization Cristea-Platon, Tudor Mazoure, Bogdan Susskind, Josh Talbott, Walter Machine Learning Artificial Intelligence Robotics Reinforcement learning practitioners often avoid hierarchical policies, especially in image-based observation spaces. Typically, the single-task performance improvement over flat-policy counterparts does not justify the additional complexity associated with implementing a hierarchy. However, by introducing multiple decision-making levels, hierarchical policies can compose lower-level policies to more effectively generalize between tasks, highlighting the need for multi-task evaluations. We analyze the benefits of hierarchy through simulated multi-task robotic control experiments from pixels. Our results show that hierarchical policies trained with task conditioning can (1) increase performance on training tasks, (2) lead to improved reward and state-space generalizations in similar tasks, and (3) decrease the complexity of fine tuning required to solve novel tasks. Thus, we believe that hierarchical policies should be considered when building reinforcement learning architectures capable of generalizing between tasks. |
| title | On the benefits of pixel-based hierarchical policies for task generalization |
| topic | Machine Learning Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2407.19142 |