SENTINEL: A Fully End-to-End Language-Action Model for Humanoid Whole Body Control
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915635262390272 |
|---|---|
| author | Wang, Yuxuan Jiang, Haobin Yao, Shiqing Ding, Ziluo Lu, Zongqing |
| author_facet | Wang, Yuxuan Jiang, Haobin Yao, Shiqing Ding, Ziluo Lu, Zongqing |
| contents | Existing humanoid control systems often rely on teleoperation or modular generation pipelines that separate language understanding from physical execution. However, the former is entirely human-driven, and the latter lacks tight alignment between language commands and physical behaviors. In this paper, we present SENTINEL, a fully end-to-end language-action model for humanoid whole-body control. We construct a large-scale dataset by tracking human motions in simulation using a pretrained whole body controller, combined with their text annotations. The model directly maps language commands and proprioceptive inputs to low-level actions without any intermediate representation. The model generates action chunks using flow matching, which can be subsequently refined by a residual action head for real-world deployment. Our method exhibits strong semantic understanding and stable execution on humanoid robots in both simulation and real-world deployment, and also supports multi-modal extensions by converting inputs into texts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_19236 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SENTINEL: A Fully End-to-End Language-Action Model for Humanoid Whole Body Control Wang, Yuxuan Jiang, Haobin Yao, Shiqing Ding, Ziluo Lu, Zongqing Robotics Artificial Intelligence Existing humanoid control systems often rely on teleoperation or modular generation pipelines that separate language understanding from physical execution. However, the former is entirely human-driven, and the latter lacks tight alignment between language commands and physical behaviors. In this paper, we present SENTINEL, a fully end-to-end language-action model for humanoid whole-body control. We construct a large-scale dataset by tracking human motions in simulation using a pretrained whole body controller, combined with their text annotations. The model directly maps language commands and proprioceptive inputs to low-level actions without any intermediate representation. The model generates action chunks using flow matching, which can be subsequently refined by a residual action head for real-world deployment. Our method exhibits strong semantic understanding and stable execution on humanoid robots in both simulation and real-world deployment, and also supports multi-modal extensions by converting inputs into texts. |
| title | SENTINEL: A Fully End-to-End Language-Action Model for Humanoid Whole Body Control |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2511.19236 |