Do You Have Freestyle? Expressive Humanoid Locomotion via Audio Control
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914232677695488 |
|---|---|
| author | Li, Zhe Chi, Cheng Wei, Yangyang Zhu, Boan Huang, Tao Sun, Zhenguo Peng, Yibo Wang, Pengwei Wang, Zhongyuan Liu, Fangzhou Xu, Chang Zhang, Shanghang |
| author_facet | Li, Zhe Chi, Cheng Wei, Yangyang Zhu, Boan Huang, Tao Sun, Zhenguo Peng, Yibo Wang, Pengwei Wang, Zhongyuan Liu, Fangzhou Xu, Chang Zhang, Shanghang |
| contents | Humans intuitively move to sound, but current humanoid robots lack expressive improvisational capabilities, confined to predefined motions or sparse commands. Generating motion from audio and then retargeting it to robots relies on explicit motion reconstruction, leading to cascaded errors, high latency, and disjointed acoustic-actuation mapping. We propose RoboPerform, the first unified audio-to-locomotion framework that can directly generate music-driven dance and speech-driven co-speech gestures from audio. Guided by the core principle of "motion = content + style", the framework treats audio as implicit style signals and eliminates the need for explicit motion reconstruction. RoboPerform integrates a ResMoE teacher policy for adapting to diverse motion patterns and a diffusion-based student policy for audio style injection. This retargeting-free design ensures low latency and high fidelity. Experimental validation shows that RoboPerform achieves promising results in physical plausibility and audio alignment, successfully transforming robots into responsive performers capable of reacting to audio. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_23650 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Do You Have Freestyle? Expressive Humanoid Locomotion via Audio Control Li, Zhe Chi, Cheng Wei, Yangyang Zhu, Boan Huang, Tao Sun, Zhenguo Peng, Yibo Wang, Pengwei Wang, Zhongyuan Liu, Fangzhou Xu, Chang Zhang, Shanghang Robotics Humans intuitively move to sound, but current humanoid robots lack expressive improvisational capabilities, confined to predefined motions or sparse commands. Generating motion from audio and then retargeting it to robots relies on explicit motion reconstruction, leading to cascaded errors, high latency, and disjointed acoustic-actuation mapping. We propose RoboPerform, the first unified audio-to-locomotion framework that can directly generate music-driven dance and speech-driven co-speech gestures from audio. Guided by the core principle of "motion = content + style", the framework treats audio as implicit style signals and eliminates the need for explicit motion reconstruction. RoboPerform integrates a ResMoE teacher policy for adapting to diverse motion patterns and a diffusion-based student policy for audio style injection. This retargeting-free design ensures low latency and high fidelity. Experimental validation shows that RoboPerform achieves promising results in physical plausibility and audio alignment, successfully transforming robots into responsive performers capable of reacting to audio. |
| title | Do You Have Freestyle? Expressive Humanoid Locomotion via Audio Control |
| topic | Robotics |
| url | https://arxiv.org/abs/2512.23650 |