Umbrella Reinforcement Learning -- computationally efficient tool for hard non-linear problems
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912249294094336 |
|---|---|
| author | Nuzhin, Egor E. Brilliantov, Nikolai V. |
| author_facet | Nuzhin, Egor E. Brilliantov, Nikolai V. |
| contents | We report a novel, computationally efficient approach for solving hard nonlinear problems of reinforcement learning (RL). Here we combine umbrella sampling, from computational physics/chemistry, with optimal control methods. The approach is realized on the basis of neural networks, with the use of policy gradient. It outperforms, by computational efficiency and implementation universality, all available state-of-the-art algorithms, in application to hard RL problems with sparse reward, state traps and lack of terminal states. The proposed approach uses an ensemble of simultaneously acting agents, with a modified reward which includes the ensemble entropy, yielding an optimal exploration-exploitation balance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_14117 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Umbrella Reinforcement Learning -- computationally efficient tool for hard non-linear problems Nuzhin, Egor E. Brilliantov, Nikolai V. Machine Learning Artificial Intelligence I.2.6; I.2.8 We report a novel, computationally efficient approach for solving hard nonlinear problems of reinforcement learning (RL). Here we combine umbrella sampling, from computational physics/chemistry, with optimal control methods. The approach is realized on the basis of neural networks, with the use of policy gradient. It outperforms, by computational efficiency and implementation universality, all available state-of-the-art algorithms, in application to hard RL problems with sparse reward, state traps and lack of terminal states. The proposed approach uses an ensemble of simultaneously acting agents, with a modified reward which includes the ensemble entropy, yielding an optimal exploration-exploitation balance. |
| title | Umbrella Reinforcement Learning -- computationally efficient tool for hard non-linear problems |
| topic | Machine Learning Artificial Intelligence I.2.6; I.2.8 |
| url | https://arxiv.org/abs/2411.14117 |