EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911267168452608 |
|---|---|
| author | Huang, Siyuan Chen, Liliang Zhou, Pengfei Chen, Shengcong Jiang, Zhengkai Hu, Yue Liao, Yue Gao, Peng Li, Hongsheng Yao, Maoqing Ren, Guanghui |
| author_facet | Huang, Siyuan Chen, Liliang Zhou, Pengfei Chen, Shengcong Jiang, Zhengkai Hu, Yue Liao, Yue Gao, Peng Li, Hongsheng Yao, Maoqing Ren, Guanghui |
| contents | We introduce EnerVerse, a generative robotics foundation model that constructs and interprets embodied spaces. EnerVerse employs a chunk-wise autoregressive video diffusion framework to predict future embodied spaces from instructions, enhanced by a sparse context memory for long-term reasoning. To model the 3D robotics world, we adopt a multi-view video representation, providing rich perspectives to address challenges like motion ambiguity and 3D grounding. Additionally, EnerVerse-D, a data engine pipeline combining generative modeling with 4D Gaussian Splatting, forms a self-reinforcing data loop to reduce the sim-to-real gap. Leveraging these innovations, EnerVerse translates 4D world representations into physical actions via a policy head (EnerVerse-A), achieving state-of-the-art performance in both simulation and real-world tasks. For efficiency, EnerVerse-A reuses features from the first denoising step and predicts action chunks, achieving about 280 ms per 8-step action chunk on a single RTX 4090. Further video demos, dataset samples could be found in our project page. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_01895 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation Huang, Siyuan Chen, Liliang Zhou, Pengfei Chen, Shengcong Jiang, Zhengkai Hu, Yue Liao, Yue Gao, Peng Li, Hongsheng Yao, Maoqing Ren, Guanghui Robotics Computer Vision and Pattern Recognition Machine Learning We introduce EnerVerse, a generative robotics foundation model that constructs and interprets embodied spaces. EnerVerse employs a chunk-wise autoregressive video diffusion framework to predict future embodied spaces from instructions, enhanced by a sparse context memory for long-term reasoning. To model the 3D robotics world, we adopt a multi-view video representation, providing rich perspectives to address challenges like motion ambiguity and 3D grounding. Additionally, EnerVerse-D, a data engine pipeline combining generative modeling with 4D Gaussian Splatting, forms a self-reinforcing data loop to reduce the sim-to-real gap. Leveraging these innovations, EnerVerse translates 4D world representations into physical actions via a policy head (EnerVerse-A), achieving state-of-the-art performance in both simulation and real-world tasks. For efficiency, EnerVerse-A reuses features from the first denoising step and predicts action chunks, achieving about 280 ms per 8-step action chunk on a single RTX 4090. Further video demos, dataset samples could be found in our project page. |
| title | EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation |
| topic | Robotics Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2501.01895 |