ReALFRED: An Embodied Instruction Following Benchmark in Photo-Realistic Environments
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910543473803264 |
|---|---|
| author | Kim, Taewoong Min, Cheolhong Kim, Byeonghwi Kim, Jinyeon Jeung, Wonje Choi, Jonghyun |
| author_facet | Kim, Taewoong Min, Cheolhong Kim, Byeonghwi Kim, Jinyeon Jeung, Wonje Choi, Jonghyun |
| contents | Simulated virtual environments have been widely used to learn robotic agents that perform daily household tasks. These environments encourage research progress by far, but often provide limited object interactability, visual appearance different from real-world environments, or relatively smaller environment sizes. This prevents the learned models in the virtual scenes from being readily deployable. To bridge the gap between these learning environments and deploying (i.e., real) environments, we propose the ReALFRED benchmark that employs real-world scenes, objects, and room layouts to learn agents to complete household tasks by understanding free-form language instructions and interacting with objects in large, multi-room and 3D-captured scenes. Specifically, we extend the ALFRED benchmark with updates for larger environmental spaces with smaller visual domain gaps. With ReALFRED, we analyze previously crafted methods for the ALFRED benchmark and observe that they consistently yield lower performance in all metrics, encouraging the community to develop methods in more realistic environments. Our code and data are publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_18550 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ReALFRED: An Embodied Instruction Following Benchmark in Photo-Realistic Environments Kim, Taewoong Min, Cheolhong Kim, Byeonghwi Kim, Jinyeon Jeung, Wonje Choi, Jonghyun Robotics Artificial Intelligence Simulated virtual environments have been widely used to learn robotic agents that perform daily household tasks. These environments encourage research progress by far, but often provide limited object interactability, visual appearance different from real-world environments, or relatively smaller environment sizes. This prevents the learned models in the virtual scenes from being readily deployable. To bridge the gap between these learning environments and deploying (i.e., real) environments, we propose the ReALFRED benchmark that employs real-world scenes, objects, and room layouts to learn agents to complete household tasks by understanding free-form language instructions and interacting with objects in large, multi-room and 3D-captured scenes. Specifically, we extend the ALFRED benchmark with updates for larger environmental spaces with smaller visual domain gaps. With ReALFRED, we analyze previously crafted methods for the ALFRED benchmark and observe that they consistently yield lower performance in all metrics, encouraging the community to develop methods in more realistic environments. Our code and data are publicly available. |
| title | ReALFRED: An Embodied Instruction Following Benchmark in Photo-Realistic Environments |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2407.18550 |