Language-guided Skill Learning with Temporal Variational Inference
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917675524947968 |
|---|---|
| author | Fu, Haotian Sharma, Pratyusha Stengel-Eskin, Elias Konidaris, George Roux, Nicolas Le Côté, Marc-Alexandre Yuan, Xingdi |
| author_facet | Fu, Haotian Sharma, Pratyusha Stengel-Eskin, Elias Konidaris, George Roux, Nicolas Le Côté, Marc-Alexandre Yuan, Xingdi |
| contents | We present an algorithm for skill discovery from expert demonstrations. The algorithm first utilizes Large Language Models (LLMs) to propose an initial segmentation of the trajectories. Following that, a hierarchical variational inference framework incorporates the LLM-generated segmentation information to discover reusable skills by merging trajectory segments. To further control the trade-off between compression and reusability, we introduce a novel auxiliary objective based on the Minimum Description Length principle that helps guide this skill discovery process. Our results demonstrate that agents equipped with our method are able to discover skills that help accelerate learning and outperform baseline skill learning approaches on new long-horizon tasks in BabyAI, a grid world navigation environment, as well as ALFRED, a household simulation environment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_16354 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Language-guided Skill Learning with Temporal Variational Inference Fu, Haotian Sharma, Pratyusha Stengel-Eskin, Elias Konidaris, George Roux, Nicolas Le Côté, Marc-Alexandre Yuan, Xingdi Machine Learning Artificial Intelligence Computation and Language We present an algorithm for skill discovery from expert demonstrations. The algorithm first utilizes Large Language Models (LLMs) to propose an initial segmentation of the trajectories. Following that, a hierarchical variational inference framework incorporates the LLM-generated segmentation information to discover reusable skills by merging trajectory segments. To further control the trade-off between compression and reusability, we introduce a novel auxiliary objective based on the Minimum Description Length principle that helps guide this skill discovery process. Our results demonstrate that agents equipped with our method are able to discover skills that help accelerate learning and outperform baseline skill learning approaches on new long-horizon tasks in BabyAI, a grid world navigation environment, as well as ALFRED, a household simulation environment. |
| title | Language-guided Skill Learning with Temporal Variational Inference |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2402.16354 |