A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916326513049600 |
|---|---|
| author | Almuzairee, Abdulaziz Hansen, Nicklas Christensen, Henrik I. |
| author_facet | Almuzairee, Abdulaziz Hansen, Nicklas Christensen, Henrik I. |
| contents | Q-learning algorithms are appealing for real-world applications due to their data-efficiency, but they are very prone to overfitting and training instabilities when trained from visual observations. Prior work, namely SVEA, finds that selective application of data augmentation can improve the visual generalization of RL agents without destabilizing training. We revisit its recipe for data augmentation, and find an assumption that limits its effectiveness to augmentations of a photometric nature. Addressing these limitations, we propose a generalized recipe, SADA, that works with wider varieties of augmentations. We benchmark its effectiveness on DMC-GB2 - our proposed extension of the popular DMControl Generalization Benchmark - as well as tasks from Meta-World and the Distracting Control Suite, and find that our method, SADA, greatly improves training stability and generalization of RL agents across a diverse set of augmentations. For visualizations, code and benchmark: see https://aalmuzairee.github.io/SADA/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_17416 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning Almuzairee, Abdulaziz Hansen, Nicklas Christensen, Henrik I. Machine Learning Computer Vision and Pattern Recognition Robotics Q-learning algorithms are appealing for real-world applications due to their data-efficiency, but they are very prone to overfitting and training instabilities when trained from visual observations. Prior work, namely SVEA, finds that selective application of data augmentation can improve the visual generalization of RL agents without destabilizing training. We revisit its recipe for data augmentation, and find an assumption that limits its effectiveness to augmentations of a photometric nature. Addressing these limitations, we propose a generalized recipe, SADA, that works with wider varieties of augmentations. We benchmark its effectiveness on DMC-GB2 - our proposed extension of the popular DMControl Generalization Benchmark - as well as tasks from Meta-World and the Distracting Control Suite, and find that our method, SADA, greatly improves training stability and generalization of RL agents across a diverse set of augmentations. For visualizations, code and benchmark: see https://aalmuzairee.github.io/SADA/ |
| title | A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning |
| topic | Machine Learning Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2405.17416 |