Simulating Infant First-Person Sensorimotor Experience via Motion Retargeting from Babies to Humanoids
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917450529898496 |
|---|---|
| author | López, Francisco M. Kanazawa, Hoshinori Fiala, Ondrej Balashov, Yakov Marcel, Valentin Rustler, Lukas Lenz, Miles Kim, Dongmin Kuniyoshi, Yasuo Triesch, Jochen Hoffmann, Matej |
| author_facet | López, Francisco M. Kanazawa, Hoshinori Fiala, Ondrej Balashov, Yakov Marcel, Valentin Rustler, Lukas Lenz, Miles Kim, Dongmin Kuniyoshi, Yasuo Triesch, Jochen Hoffmann, Matej |
| contents | Motion retargeting from humans to human-like artificial agents is becoming increasingly important as humanoid robots grow more capable. However, most existing approaches focus only on reproducing kinematics and ignore the rich sensorimotor experience associated with human movement. In this work, we present a framework for simulating the multimodal sensorimotor experiences of infants using physical and virtual humanoids. From a single video, our method reconstructs the infant's body configuration by extracting its skeletal structure and estimating the full 3D pose from each frame. Then we map the reconstructed motion onto several developmental platforms: the physical iCub robot and the virtual simulators pyCub, EMFANT and MIMo. Replaying the retargeted motions on these embodiments produces simulated multisensory streams including proprioception (joints and muscles), touch, and vision. For the best-matching embodiment, the retargeting achieves sub-centimeter accuracy and enables a rich multimodal analysis of infant development as well as enhanced automated annotation of behaviors. This framework provides a unique window into the infant's sensorimotor experience, offering new tools for robotics, developmental science, and early detection of neurodevelopmental disorders. The code is available at https://github.com/ctu-vras/motion-retargeting/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_27583 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Simulating Infant First-Person Sensorimotor Experience via Motion Retargeting from Babies to Humanoids López, Francisco M. Kanazawa, Hoshinori Fiala, Ondrej Balashov, Yakov Marcel, Valentin Rustler, Lukas Lenz, Miles Kim, Dongmin Kuniyoshi, Yasuo Triesch, Jochen Hoffmann, Matej Neurons and Cognition Robotics Motion retargeting from humans to human-like artificial agents is becoming increasingly important as humanoid robots grow more capable. However, most existing approaches focus only on reproducing kinematics and ignore the rich sensorimotor experience associated with human movement. In this work, we present a framework for simulating the multimodal sensorimotor experiences of infants using physical and virtual humanoids. From a single video, our method reconstructs the infant's body configuration by extracting its skeletal structure and estimating the full 3D pose from each frame. Then we map the reconstructed motion onto several developmental platforms: the physical iCub robot and the virtual simulators pyCub, EMFANT and MIMo. Replaying the retargeted motions on these embodiments produces simulated multisensory streams including proprioception (joints and muscles), touch, and vision. For the best-matching embodiment, the retargeting achieves sub-centimeter accuracy and enables a rich multimodal analysis of infant development as well as enhanced automated annotation of behaviors. This framework provides a unique window into the infant's sensorimotor experience, offering new tools for robotics, developmental science, and early detection of neurodevelopmental disorders. The code is available at https://github.com/ctu-vras/motion-retargeting/. |
| title | Simulating Infant First-Person Sensorimotor Experience via Motion Retargeting from Babies to Humanoids |
| topic | Neurons and Cognition Robotics |
| url | https://arxiv.org/abs/2604.27583 |