Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2605.26371 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918530990997504 |
|---|---|
| author | Dayal, Sarthak Peri, Abhinav Qi, Carl Voelcker, Claas Levine, Alexander Chuck, Caleb Zhang, Amy |
| author_facet | Dayal, Sarthak Peri, Abhinav Qi, Carl Voelcker, Claas Levine, Alexander Chuck, Caleb Zhang, Amy |
| contents | Hierarchical Reinforcement Learning (HRL) promises to solve long-horizon Reinforcement Learning (RL) tasks more efficiently than non-hierarchical counterparts by discovering and reusing temporally-extended skills. However, obtaining skills that are actually reusable remains an open challenge. Towards this end, we focus on abstractions that exploit the intuition of local dynamics: local transitions in different global contexts require similar kinds of action sequences. By aligning these contexts with the action sequences they require, we are able to learn which skills to reuse and where to reuse them. In principle, this information should benefit many HRL algorithms, where high-level policies have to reason about the low-level skills they use. The resulting algorithm CARL (Contrastive Action-based Representations for Reusable Local Control) shows both qualitative clustering of meaningful skills in complex humanoid environments and improved downstream performance on the OGBench benchmark when integrated with HIQL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_26371 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Exploiting Local Dynamics Regularity for Reusable Skills in Offline Hierarchical RL Dayal, Sarthak Peri, Abhinav Qi, Carl Voelcker, Claas Levine, Alexander Chuck, Caleb Zhang, Amy Artificial Intelligence Hierarchical Reinforcement Learning (HRL) promises to solve long-horizon Reinforcement Learning (RL) tasks more efficiently than non-hierarchical counterparts by discovering and reusing temporally-extended skills. However, obtaining skills that are actually reusable remains an open challenge. Towards this end, we focus on abstractions that exploit the intuition of local dynamics: local transitions in different global contexts require similar kinds of action sequences. By aligning these contexts with the action sequences they require, we are able to learn which skills to reuse and where to reuse them. In principle, this information should benefit many HRL algorithms, where high-level policies have to reason about the low-level skills they use. The resulting algorithm CARL (Contrastive Action-based Representations for Reusable Local Control) shows both qualitative clustering of meaningful skills in complex humanoid environments and improved downstream performance on the OGBench benchmark when integrated with HIQL. |
| title | Exploiting Local Dynamics Regularity for Reusable Skills in Offline Hierarchical RL |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2605.26371 |