Equivariant Action Sampling for Reinforcement Learning and Planning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915066890158080 |
|---|---|
| author | Zhao, Linfeng Howell, Owen Zhu, Xupeng Park, Jung Yeon Zhang, Zhewen Walters, Robin Wong, Lawson L. S. |
| author_facet | Zhao, Linfeng Howell, Owen Zhu, Xupeng Park, Jung Yeon Zhang, Zhewen Walters, Robin Wong, Lawson L. S. |
| contents | Reinforcement learning (RL) algorithms for continuous control tasks require accurate sampling-based action selection. Many tasks, such as robotic manipulation, contain inherent problem symmetries. However, correctly incorporating symmetry into sampling-based approaches remains a challenge. This work addresses the challenge of preserving symmetry in sampling-based planning and control, a key component for enhancing decision-making efficiency in RL. We introduce an action sampling approach that enforces the desired symmetry. We apply our proposed method to a coordinate regression problem and show that the symmetry aware sampling method drastically outperforms the naive sampling approach. We furthermore develop a general framework for sampling-based model-based planning with Model Predictive Path Integral (MPPI). We compare our MPPI approach with standard sampling methods on several continuous control tasks. Empirical demonstrations across multiple continuous control environments validate the effectiveness of our approach, showcasing the importance of symmetry preservation in sampling-based action selection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_12237 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Equivariant Action Sampling for Reinforcement Learning and Planning Zhao, Linfeng Howell, Owen Zhu, Xupeng Park, Jung Yeon Zhang, Zhewen Walters, Robin Wong, Lawson L. S. Robotics Artificial Intelligence Machine Learning Reinforcement learning (RL) algorithms for continuous control tasks require accurate sampling-based action selection. Many tasks, such as robotic manipulation, contain inherent problem symmetries. However, correctly incorporating symmetry into sampling-based approaches remains a challenge. This work addresses the challenge of preserving symmetry in sampling-based planning and control, a key component for enhancing decision-making efficiency in RL. We introduce an action sampling approach that enforces the desired symmetry. We apply our proposed method to a coordinate regression problem and show that the symmetry aware sampling method drastically outperforms the naive sampling approach. We furthermore develop a general framework for sampling-based model-based planning with Model Predictive Path Integral (MPPI). We compare our MPPI approach with standard sampling methods on several continuous control tasks. Empirical demonstrations across multiple continuous control environments validate the effectiveness of our approach, showcasing the importance of symmetry preservation in sampling-based action selection. |
| title | Equivariant Action Sampling for Reinforcement Learning and Planning |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2412.12237 |