UniAct: Unified Motion Generation and Action Streaming for Humanoid Robots
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912797231677440 |
|---|---|
| author | Jiang, Nan He, Zimo Yu, Wanhe Pang, Lexi Li, Yunhao Li, Hongjie Cui, Jieming Li, Yuhan Wang, Yizhou Zhu, Yixin Huang, Siyuan |
| author_facet | Jiang, Nan He, Zimo Yu, Wanhe Pang, Lexi Li, Yunhao Li, Hongjie Cui, Jieming Li, Yuhan Wang, Yizhou Zhu, Yixin Huang, Siyuan |
| contents | A long-standing objective in humanoid robotics is the realization of versatile agents capable of following diverse multimodal instructions with human-level flexibility. Despite advances in humanoid control, bridging high-level multimodal perception with whole-body execution remains a significant bottleneck. Existing methods often struggle to translate heterogeneous instructions -- such as language, music, and trajectories -- into stable, real-time actions. Here we show that UniAct, a two-stage framework integrating a fine-tuned MLLM with a causal streaming pipeline, enables humanoid robots to execute multimodal instructions with sub-500 ms latency. By unifying inputs through a shared discrete codebook via FSQ, UniAct ensures cross-modal alignment while constraining motions to a physically grounded manifold. This approach yields a 19% improvement in the success rate of zero-shot tracking of imperfect reference motions. We validate UniAct on UniMoCap, our 20-hour humanoid motion benchmark, demonstrating robust generalization across diverse real-world scenarios. Our results mark a critical step toward responsive, general-purpose humanoid assistants capable of seamless interaction through unified perception and control. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_24321 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UniAct: Unified Motion Generation and Action Streaming for Humanoid Robots Jiang, Nan He, Zimo Yu, Wanhe Pang, Lexi Li, Yunhao Li, Hongjie Cui, Jieming Li, Yuhan Wang, Yizhou Zhu, Yixin Huang, Siyuan Computer Vision and Pattern Recognition Robotics A long-standing objective in humanoid robotics is the realization of versatile agents capable of following diverse multimodal instructions with human-level flexibility. Despite advances in humanoid control, bridging high-level multimodal perception with whole-body execution remains a significant bottleneck. Existing methods often struggle to translate heterogeneous instructions -- such as language, music, and trajectories -- into stable, real-time actions. Here we show that UniAct, a two-stage framework integrating a fine-tuned MLLM with a causal streaming pipeline, enables humanoid robots to execute multimodal instructions with sub-500 ms latency. By unifying inputs through a shared discrete codebook via FSQ, UniAct ensures cross-modal alignment while constraining motions to a physically grounded manifold. This approach yields a 19% improvement in the success rate of zero-shot tracking of imperfect reference motions. We validate UniAct on UniMoCap, our 20-hour humanoid motion benchmark, demonstrating robust generalization across diverse real-world scenarios. Our results mark a critical step toward responsive, general-purpose humanoid assistants capable of seamless interaction through unified perception and control. |
| title | UniAct: Unified Motion Generation and Action Streaming for Humanoid Robots |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2512.24321 |