Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910434915778560 |
|---|---|
| author | Tirumala, Dhruva Wulfmeier, Markus Moran, Ben Huang, Sandy Humplik, Jan Lever, Guy Haarnoja, Tuomas Hasenclever, Leonard Byravan, Arunkumar Batchelor, Nathan Sreendra, Neil Patel, Kushal Gwira, Marlon Nori, Francesco Riedmiller, Martin Heess, Nicolas |
| author_facet | Tirumala, Dhruva Wulfmeier, Markus Moran, Ben Huang, Sandy Humplik, Jan Lever, Guy Haarnoja, Tuomas Hasenclever, Leonard Byravan, Arunkumar Batchelor, Nathan Sreendra, Neil Patel, Kushal Gwira, Marlon Nori, Francesco Riedmiller, Martin Heess, Nicolas |
| contents | We apply multi-agent deep reinforcement learning (RL) to train end-to-end robot soccer policies with fully onboard computation and sensing via egocentric RGB vision. This setting reflects many challenges of real-world robotics, including active perception, agile full-body control, and long-horizon planning in a dynamic, partially-observable, multi-agent domain. We rely on large-scale, simulation-based data generation to obtain complex behaviors from egocentric vision which can be successfully transferred to physical robots using low-cost sensors. To achieve adequate visual realism, our simulation combines rigid-body physics with learned, realistic rendering via multiple Neural Radiance Fields (NeRFs). We combine teacher-based multi-agent RL and cross-experiment data reuse to enable the discovery of sophisticated soccer strategies. We analyze active-perception behaviors including object tracking and ball seeking that emerge when simply optimizing perception-agnostic soccer play. The agents display equivalent levels of performance and agility as policies with access to privileged, ground-truth state. To our knowledge, this paper constitutes a first demonstration of end-to-end training for multi-agent robot soccer, mapping raw pixel observations to joint-level actions, that can be deployed in the real world. Videos of the game-play and analyses can be seen on our website https://sites.google.com/view/vision-soccer . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_02425 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning Tirumala, Dhruva Wulfmeier, Markus Moran, Ben Huang, Sandy Humplik, Jan Lever, Guy Haarnoja, Tuomas Hasenclever, Leonard Byravan, Arunkumar Batchelor, Nathan Sreendra, Neil Patel, Kushal Gwira, Marlon Nori, Francesco Riedmiller, Martin Heess, Nicolas Robotics Artificial Intelligence We apply multi-agent deep reinforcement learning (RL) to train end-to-end robot soccer policies with fully onboard computation and sensing via egocentric RGB vision. This setting reflects many challenges of real-world robotics, including active perception, agile full-body control, and long-horizon planning in a dynamic, partially-observable, multi-agent domain. We rely on large-scale, simulation-based data generation to obtain complex behaviors from egocentric vision which can be successfully transferred to physical robots using low-cost sensors. To achieve adequate visual realism, our simulation combines rigid-body physics with learned, realistic rendering via multiple Neural Radiance Fields (NeRFs). We combine teacher-based multi-agent RL and cross-experiment data reuse to enable the discovery of sophisticated soccer strategies. We analyze active-perception behaviors including object tracking and ball seeking that emerge when simply optimizing perception-agnostic soccer play. The agents display equivalent levels of performance and agility as policies with access to privileged, ground-truth state. To our knowledge, this paper constitutes a first demonstration of end-to-end training for multi-agent robot soccer, mapping raw pixel observations to joint-level actions, that can be deployed in the real world. Videos of the game-play and analyses can be seen on our website https://sites.google.com/view/vision-soccer . |
| title | Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2405.02425 |