Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909681408016384 |
|---|---|
| author | Bloesch, Michael Wulfmeier, Markus Brakel, Philemon Davchev, Todor Zambelli, Martina Springenberg, Jost Tobias Abdolmaleki, Abbas Whitney, William F Heess, Nicolas Hafner, Roland Riedmiller, Martin |
| author_facet | Bloesch, Michael Wulfmeier, Markus Brakel, Philemon Davchev, Todor Zambelli, Martina Springenberg, Jost Tobias Abdolmaleki, Abbas Whitney, William F Heess, Nicolas Hafner, Roland Riedmiller, Martin |
| contents | Imitation Learning from Observation (IfO) offers a powerful way to learn behaviors at large-scale: Unlike behavior cloning or offline reinforcement learning, IfO can leverage action-free demonstrations and thus circumvents the need for costly action-labeled demonstrations or reward functions. However, current IfO research focuses on idealized scenarios with mostly bimodal-quality data distributions, restricting the meaningfulness of the results. In contrast, this paper investigates more nuanced distributions and introduces a method to learn from such data, moving closer to a paradigm in which imitation learning can be performed iteratively via self-improvement. Our method adapts RL-based imitation learning to action-free demonstrations, using a value function to transfer information between expert and non-expert data. Through comprehensive evaluation, we delineate the relation between different data distributions and the applicability of algorithms and highlight the limitations of established methods. Our findings provide valuable insights for developing more robust and practical IfO techniques on a path to scalable behaviour learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_06701 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement Bloesch, Michael Wulfmeier, Markus Brakel, Philemon Davchev, Todor Zambelli, Martina Springenberg, Jost Tobias Abdolmaleki, Abbas Whitney, William F Heess, Nicolas Hafner, Roland Riedmiller, Martin Machine Learning Imitation Learning from Observation (IfO) offers a powerful way to learn behaviors at large-scale: Unlike behavior cloning or offline reinforcement learning, IfO can leverage action-free demonstrations and thus circumvents the need for costly action-labeled demonstrations or reward functions. However, current IfO research focuses on idealized scenarios with mostly bimodal-quality data distributions, restricting the meaningfulness of the results. In contrast, this paper investigates more nuanced distributions and introduces a method to learn from such data, moving closer to a paradigm in which imitation learning can be performed iteratively via self-improvement. Our method adapts RL-based imitation learning to action-free demonstrations, using a value function to transfer information between expert and non-expert data. Through comprehensive evaluation, we delineate the relation between different data distributions and the applicability of algorithms and highlight the limitations of established methods. Our findings provide valuable insights for developing more robust and practical IfO techniques on a path to scalable behaviour learning. |
| title | Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2507.06701 |