Making Avatars Interact: Towards Text-Driven Human-Object Interaction for Controllable Talking Avatars
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912870547062784 |
|---|---|
| author | Zhang, Youliang Zhou, Zhengguang Yu, Zhentao Huang, Ziyao Hu, Teng Liang, Sen Zhang, Guozhen Peng, Ziqiao Li, Shunkai Chen, Yi Zhou, Zixiang Zhou, Yuan Lu, Qinglin Li, Xiu |
| author_facet | Zhang, Youliang Zhou, Zhengguang Yu, Zhentao Huang, Ziyao Hu, Teng Liang, Sen Zhang, Guozhen Peng, Ziqiao Li, Shunkai Chen, Yi Zhou, Zixiang Zhou, Yuan Lu, Qinglin Li, Xiu |
| contents | Generating talking avatars is a fundamental task in video generation. Although existing methods can generate full-body talking avatars with simple human motion, extending this task to grounded human-object interaction (GHOI) remains an open challenge, requiring the avatar to perform text-aligned interactions with surrounding objects. This challenge stems from the need for environmental perception and the control-quality dilemma in GHOI generation. To address this, we propose a novel dual-stream framework, InteractAvatar, which decouples perception and planning from video synthesis for grounded human-object interaction. Leveraging detection to enhance environmental perception, we introduce a Perception and Interaction Module (PIM) to generate text-aligned interaction motions. Additionally, an Audio-Interaction Aware Generation Module (AIM) is proposed to synthesize vivid talking avatars performing object interactions. With a specially designed motion-to-video aligner, PIM and AIM share a similar network structure and enable parallel co-generation of motions and plausible videos, effectively mitigating the control-quality dilemma. Finally, we establish a benchmark, GroundedInter, for evaluating GHOI video generation. Extensive experiments and comparisons demonstrate the effectiveness of our method in generating grounded human-object interactions for talking avatars. Project page: https://interactavatar.github.io |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_01538 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Making Avatars Interact: Towards Text-Driven Human-Object Interaction for Controllable Talking Avatars Zhang, Youliang Zhou, Zhengguang Yu, Zhentao Huang, Ziyao Hu, Teng Liang, Sen Zhang, Guozhen Peng, Ziqiao Li, Shunkai Chen, Yi Zhou, Zixiang Zhou, Yuan Lu, Qinglin Li, Xiu Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Generating talking avatars is a fundamental task in video generation. Although existing methods can generate full-body talking avatars with simple human motion, extending this task to grounded human-object interaction (GHOI) remains an open challenge, requiring the avatar to perform text-aligned interactions with surrounding objects. This challenge stems from the need for environmental perception and the control-quality dilemma in GHOI generation. To address this, we propose a novel dual-stream framework, InteractAvatar, which decouples perception and planning from video synthesis for grounded human-object interaction. Leveraging detection to enhance environmental perception, we introduce a Perception and Interaction Module (PIM) to generate text-aligned interaction motions. Additionally, an Audio-Interaction Aware Generation Module (AIM) is proposed to synthesize vivid talking avatars performing object interactions. With a specially designed motion-to-video aligner, PIM and AIM share a similar network structure and enable parallel co-generation of motions and plausible videos, effectively mitigating the control-quality dilemma. Finally, we establish a benchmark, GroundedInter, for evaluating GHOI video generation. Extensive experiments and comparisons demonstrate the effectiveness of our method in generating grounded human-object interactions for talking avatars. Project page: https://interactavatar.github.io |
| title | Making Avatars Interact: Towards Text-Driven Human-Object Interaction for Controllable Talking Avatars |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2602.01538 |