Populate-A-Scene: Affordance-Aware Human Video Generation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916818199773184 |
|---|---|
| author | Shan, Mengyi He, Zecheng Ma, Haoyu Juefei-Xu, Felix Zhang, Peizhao Hou, Tingbo Chuang, Ching-Yao |
| author_facet | Shan, Mengyi He, Zecheng Ma, Haoyu Juefei-Xu, Felix Zhang, Peizhao Hou, Tingbo Chuang, Ching-Yao |
| contents | Can a video generation model be repurposed as an interactive world simulator? We explore the affordance perception potential of text-to-video models by teaching them to predict human-environment interaction. Given a scene image and a prompt describing human actions, we fine-tune the model to insert a person into the scene, while ensuring coherent behavior, appearance, harmonization, and scene affordance. Unlike prior work, we infer human affordance for video generation (i.e., where to insert a person and how they should behave) from a single scene image, without explicit conditions like bounding boxes or body poses. An in-depth study of cross-attention heatmaps demonstrates that we can uncover the inherent affordance perception of a pre-trained video model without labeled affordance datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_00334 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Populate-A-Scene: Affordance-Aware Human Video Generation Shan, Mengyi He, Zecheng Ma, Haoyu Juefei-Xu, Felix Zhang, Peizhao Hou, Tingbo Chuang, Ching-Yao Computer Vision and Pattern Recognition Can a video generation model be repurposed as an interactive world simulator? We explore the affordance perception potential of text-to-video models by teaching them to predict human-environment interaction. Given a scene image and a prompt describing human actions, we fine-tune the model to insert a person into the scene, while ensuring coherent behavior, appearance, harmonization, and scene affordance. Unlike prior work, we infer human affordance for video generation (i.e., where to insert a person and how they should behave) from a single scene image, without explicit conditions like bounding boxes or body poses. An in-depth study of cross-attention heatmaps demonstrates that we can uncover the inherent affordance perception of a pre-trained video model without labeled affordance datasets. |
| title | Populate-A-Scene: Affordance-Aware Human Video Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.00334 |