On the Benefit of Optimal Transport for Curriculum Reinforcement Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916234755309568 |
|---|---|
| author | Klink, Pascal D'Eramo, Carlo Peters, Jan Pajarinen, Joni |
| author_facet | Klink, Pascal D'Eramo, Carlo Peters, Jan Pajarinen, Joni |
| contents | Curriculum reinforcement learning (CRL) allows solving complex tasks by generating a tailored sequence of learning tasks, starting from easy ones and subsequently increasing their difficulty. Although the potential of curricula in RL has been clearly shown in various works, it is less clear how to generate them for a given learning environment, resulting in various methods aiming to automate this task. In this work, we focus on framing curricula as interpolations between task distributions, which has previously been shown to be a viable approach to CRL. Identifying key issues of existing methods, we frame the generation of a curriculum as a constrained optimal transport problem between task distributions. Benchmarks show that this way of curriculum generation can improve upon existing CRL methods, yielding high performance in various tasks with different characteristics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_14091 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | On the Benefit of Optimal Transport for Curriculum Reinforcement Learning Klink, Pascal D'Eramo, Carlo Peters, Jan Pajarinen, Joni Machine Learning Curriculum reinforcement learning (CRL) allows solving complex tasks by generating a tailored sequence of learning tasks, starting from easy ones and subsequently increasing their difficulty. Although the potential of curricula in RL has been clearly shown in various works, it is less clear how to generate them for a given learning environment, resulting in various methods aiming to automate this task. In this work, we focus on framing curricula as interpolations between task distributions, which has previously been shown to be a viable approach to CRL. Identifying key issues of existing methods, we frame the generation of a curriculum as a constrained optimal transport problem between task distributions. Benchmarks show that this way of curriculum generation can improve upon existing CRL methods, yielding high performance in various tasks with different characteristics. |
| title | On the Benefit of Optimal Transport for Curriculum Reinforcement Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2309.14091 |