Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2509.11766 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909788604989440 |
|---|---|
| author | Zhai, Andy Liu, Brae Fang, Bruno Cai, Chalse Ma, Ellie Yin, Ethan Wang, Hao Zhou, Hugo Wang, James Shi, Lights Liang, Lucy Wang, Make Wang, Qian Gan, Roy Yu, Ryan Li, Shalfun Liu, Starrick Chen, Sylas Chen, Vincent Xu, Zach |
| author_facet | Zhai, Andy Liu, Brae Fang, Bruno Cai, Chalse Ma, Ellie Yin, Ethan Wang, Hao Zhou, Hugo Wang, James Shi, Lights Liang, Lucy Wang, Make Wang, Qian Gan, Roy Yu, Ryan Li, Shalfun Liu, Starrick Chen, Sylas Chen, Vincent Xu, Zach |
| contents | While foundation models show remarkable progress in language and vision, existing vision-language models (VLMs) still have limited spatial and embodiment understanding. Transferring VLMs to embodied domains reveals fundamental mismatches between modalities, pretraining distributions, and training objectives, leaving action comprehension and generation as a central bottleneck on the path to AGI.
We introduce WALL-OSS, an end-to-end embodied foundation model that leverages large-scale multimodal pretraining to achieve (1) embodiment-aware vision-language understanding, (2) strong language-action association, and (3) robust manipulation capability.
Our approach employs a tightly coupled architecture and multi-strategies training curriculum that enables Unified Cross-Level CoT-seamlessly unifying instruction reasoning, subgoal decomposition, and fine-grained action synthesis within a single differentiable framework.
Our results show that WALL-OSS attains high success on complex long-horizon manipulations, demonstrates strong instruction-following capabilities, complex understanding and reasoning, and outperforms strong baselines, thereby providing a reliable and scalable path from VLMs to embodied foundation models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_11766 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Igniting VLMs toward the Embodied Space Zhai, Andy Liu, Brae Fang, Bruno Cai, Chalse Ma, Ellie Yin, Ethan Wang, Hao Zhou, Hugo Wang, James Shi, Lights Liang, Lucy Wang, Make Wang, Qian Gan, Roy Yu, Ryan Li, Shalfun Liu, Starrick Chen, Sylas Chen, Vincent Xu, Zach Robotics While foundation models show remarkable progress in language and vision, existing vision-language models (VLMs) still have limited spatial and embodiment understanding. Transferring VLMs to embodied domains reveals fundamental mismatches between modalities, pretraining distributions, and training objectives, leaving action comprehension and generation as a central bottleneck on the path to AGI. We introduce WALL-OSS, an end-to-end embodied foundation model that leverages large-scale multimodal pretraining to achieve (1) embodiment-aware vision-language understanding, (2) strong language-action association, and (3) robust manipulation capability. Our approach employs a tightly coupled architecture and multi-strategies training curriculum that enables Unified Cross-Level CoT-seamlessly unifying instruction reasoning, subgoal decomposition, and fine-grained action synthesis within a single differentiable framework. Our results show that WALL-OSS attains high success on complex long-horizon manipulations, demonstrates strong instruction-following capabilities, complex understanding and reasoning, and outperforms strong baselines, thereby providing a reliable and scalable path from VLMs to embodied foundation models. |
| title | Igniting VLMs toward the Embodied Space |
| topic | Robotics |
| url | https://arxiv.org/abs/2509.11766 |