Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2407.00290 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916307459375104 |
|---|---|
| author | Wang, Dong Beltrame, Giovanni |
| author_facet | Wang, Dong Beltrame, Giovanni |
| contents | Traditional reinforcement learning (RL) generates discrete control policies,
assigning one action per cycle. These policies are usually implemented as in a
fixed-frequency control loop. This rigidity presents challenges as optimal
control frequency is task-dependent; suboptimal frequencies increase
computational demands and reduce exploration efficiency. Variable Time Step
Reinforcement Learning (VTS-RL) addresses these issues with adaptive control
frequencies, executing actions only when necessary, thus reducing
computational load and extending the action space to include action durations.
In this paper we introduce the Multi-Objective Soft Elastic Actor-Critic
(MOSEAC) method to perform VTS-RL, validating it through theoretical analysis
and experimentation in simulation and on real robots. Results show faster
convergence, better training results, and reduced energy consumption with
respect to other variable- or fixed-frequency approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_00290 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Variable Time Step Reinforcement Learning for Robotic Applications Wang, Dong Beltrame, Giovanni Robotics Traditional reinforcement learning (RL) generates discrete control policies, assigning one action per cycle. These policies are usually implemented as in a fixed-frequency control loop. This rigidity presents challenges as optimal control frequency is task-dependent; suboptimal frequencies increase computational demands and reduce exploration efficiency. Variable Time Step Reinforcement Learning (VTS-RL) addresses these issues with adaptive control frequencies, executing actions only when necessary, thus reducing computational load and extending the action space to include action durations. In this paper we introduce the Multi-Objective Soft Elastic Actor-Critic (MOSEAC) method to perform VTS-RL, validating it through theoretical analysis and experimentation in simulation and on real robots. Results show faster convergence, better training results, and reduced energy consumption with respect to other variable- or fixed-frequency approaches. |
| title | Variable Time Step Reinforcement Learning for Robotic Applications |
| topic | Robotics |
| url | https://arxiv.org/abs/2407.00290 |