Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.20338 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910767071100928 |
|---|---|
| author | Zhang, Hao Wang, Hao Huang, Xiucai Chen, Wenrui Kan, Zhen |
| author_facet | Zhang, Hao Wang, Hao Huang, Xiucai Chen, Wenrui Kan, Zhen |
| contents | Reinforcement Learning (RL) based methods have been increasingly explored for robot learning. However, RL based methods often suffer from low sampling efficiency in the exploration phase, especially for long-horizon manipulation tasks, and generally neglect the semantic information from the task level, resulted in a delayed convergence or even tasks failure. To tackle these challenges, we propose a Temporal-Logic-guided Hybrid policy framework (HyTL) which leverages three-level decision layers to improve the agent's performance. Specifically, the task specifications are encoded via linear temporal logic (LTL) to improve performance and offer interpretability. And a waypoints planning module is designed with the feedback from the LTL-encoded task level as a high-level policy to improve the exploration efficiency. The middle-level policy selects which behavior primitives to execute, and the low-level policy specifies the corresponding parameters to interact with the environment. We evaluate HyTL on four challenging manipulation tasks, which demonstrate its effectiveness and interpretability. Our project is available at: https://sites.google.com/view/hytl-0257/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_20338 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploiting Hybrid Policy in Reinforcement Learning for Interpretable Temporal Logic Manipulation Zhang, Hao Wang, Hao Huang, Xiucai Chen, Wenrui Kan, Zhen Robotics Artificial Intelligence Machine Learning Reinforcement Learning (RL) based methods have been increasingly explored for robot learning. However, RL based methods often suffer from low sampling efficiency in the exploration phase, especially for long-horizon manipulation tasks, and generally neglect the semantic information from the task level, resulted in a delayed convergence or even tasks failure. To tackle these challenges, we propose a Temporal-Logic-guided Hybrid policy framework (HyTL) which leverages three-level decision layers to improve the agent's performance. Specifically, the task specifications are encoded via linear temporal logic (LTL) to improve performance and offer interpretability. And a waypoints planning module is designed with the feedback from the LTL-encoded task level as a high-level policy to improve the exploration efficiency. The middle-level policy selects which behavior primitives to execute, and the low-level policy specifies the corresponding parameters to interact with the environment. We evaluate HyTL on four challenging manipulation tasks, which demonstrate its effectiveness and interpretability. Our project is available at: https://sites.google.com/view/hytl-0257/. |
| title | Exploiting Hybrid Policy in Reinforcement Learning for Interpretable Temporal Logic Manipulation |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2412.20338 |