Equivariant Ensembles and Regularization for Reinforcement Learning in Map-based Path Planning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912170943447040 |
|---|---|
| author | Theile, Mirco Cao, Hongpeng Caccamo, Marco Sangiovanni-Vincentelli, Alberto L. |
| author_facet | Theile, Mirco Cao, Hongpeng Caccamo, Marco Sangiovanni-Vincentelli, Alberto L. |
| contents | In reinforcement learning (RL), exploiting environmental symmetries can significantly enhance efficiency, robustness, and performance. However, ensuring that the deep RL policy and value networks are respectively equivariant and invariant to exploit these symmetries is a substantial challenge. Related works try to design networks that are equivariant and invariant by construction, limiting them to a very restricted library of components, which in turn hampers the expressiveness of the networks. This paper proposes a method to construct equivariant policies and invariant value functions without specialized neural network components, which we term equivariant ensembles. We further add a regularization term for adding inductive bias during training. In a map-based path planning case study, we show how equivariant ensembles and regularization benefit sample efficiency and performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_12856 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Equivariant Ensembles and Regularization for Reinforcement Learning in Map-based Path Planning Theile, Mirco Cao, Hongpeng Caccamo, Marco Sangiovanni-Vincentelli, Alberto L. Machine Learning Robotics In reinforcement learning (RL), exploiting environmental symmetries can significantly enhance efficiency, robustness, and performance. However, ensuring that the deep RL policy and value networks are respectively equivariant and invariant to exploit these symmetries is a substantial challenge. Related works try to design networks that are equivariant and invariant by construction, limiting them to a very restricted library of components, which in turn hampers the expressiveness of the networks. This paper proposes a method to construct equivariant policies and invariant value functions without specialized neural network components, which we term equivariant ensembles. We further add a regularization term for adding inductive bias during training. In a map-based path planning case study, we show how equivariant ensembles and regularization benefit sample efficiency and performance. |
| title | Equivariant Ensembles and Regularization for Reinforcement Learning in Map-based Path Planning |
| topic | Machine Learning Robotics |
| url | https://arxiv.org/abs/2403.12856 |