A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observations
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916632252645376 |
|---|---|
| author | Ozcan, Erhan Can Giammarino, Vittorio Queeney, James Paschalidis, Ioannis Ch. |
| author_facet | Ozcan, Erhan Can Giammarino, Vittorio Queeney, James Paschalidis, Ioannis Ch. |
| contents | This paper investigates how to incorporate expert observations (without explicit information on expert actions) into a deep reinforcement learning setting to improve sample efficiency. First, we formulate an augmented policy loss combining a maximum entropy reinforcement learning objective with a behavioral cloning loss that leverages a forward dynamics model. Then, we propose an algorithm that automatically adjusts the weights of each component in the augmented loss function. Experiments on a variety of continuous control tasks demonstrate that the proposed algorithm outperforms various benchmarks by effectively utilizing available expert observations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_18836 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observations Ozcan, Erhan Can Giammarino, Vittorio Queeney, James Paschalidis, Ioannis Ch. Machine Learning This paper investigates how to incorporate expert observations (without explicit information on expert actions) into a deep reinforcement learning setting to improve sample efficiency. First, we formulate an augmented policy loss combining a maximum entropy reinforcement learning objective with a behavioral cloning loss that leverages a forward dynamics model. Then, we propose an algorithm that automatically adjusts the weights of each component in the augmented loss function. Experiments on a variety of continuous control tasks demonstrate that the proposed algorithm outperforms various benchmarks by effectively utilizing available expert observations. |
| title | A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observations |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.18836 |