Egocentric World Model for Photorealistic Hand-Object Interaction Synthesis
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915862078816256 |
|---|---|
| author | Li, Dayou Liu, Lulin Liu, Bangya Zhou, Shijie Feng, Jiu Lu, Ziqi Zheng, Minghui You, Chenyu Fan, Zhiwen |
| author_facet | Li, Dayou Liu, Lulin Liu, Bangya Zhou, Shijie Feng, Jiu Lu, Ziqi Zheng, Minghui You, Chenyu Fan, Zhiwen |
| contents | To serve as a scalable data source for embodied AI, world models should act as true simulators that infer interaction dynamics strictly from user actions, rather than mere conditional video generators relying on privileged future object states. In this context, egocentric Human-Object Interaction (HOI) world models are critical for predicting physically grounded first-person rollouts. However, building such models is profoundly challenging due to rapid head motions, severe occlusions, and high-DoF hand articulations that abruptly alter contact topologies. Consequently, existing approaches often circumvent these physics challenges by resorting to conditional video generation with access to known future object trajectories. We introduce EgoHOI, an egocentric HOI world model that breaks away from this shortcut to simulate photorealistic, contact-consistent interactions from action signals alone. To ensure physical accuracy without future-state inputs, EgoHOI distills geometric and kinematic priors from 3D estimates into physics-informed embeddings. These embeddings regularize the egocentric rollouts toward physically valid dynamics. Experiments on the HOT3D dataset demonstrate consistent gains over strong baselines, and ablations validate the effectiveness of our physics-informed design. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_13615 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Egocentric World Model for Photorealistic Hand-Object Interaction Synthesis Li, Dayou Liu, Lulin Liu, Bangya Zhou, Shijie Feng, Jiu Lu, Ziqi Zheng, Minghui You, Chenyu Fan, Zhiwen Computer Vision and Pattern Recognition Robotics To serve as a scalable data source for embodied AI, world models should act as true simulators that infer interaction dynamics strictly from user actions, rather than mere conditional video generators relying on privileged future object states. In this context, egocentric Human-Object Interaction (HOI) world models are critical for predicting physically grounded first-person rollouts. However, building such models is profoundly challenging due to rapid head motions, severe occlusions, and high-DoF hand articulations that abruptly alter contact topologies. Consequently, existing approaches often circumvent these physics challenges by resorting to conditional video generation with access to known future object trajectories. We introduce EgoHOI, an egocentric HOI world model that breaks away from this shortcut to simulate photorealistic, contact-consistent interactions from action signals alone. To ensure physical accuracy without future-state inputs, EgoHOI distills geometric and kinematic priors from 3D estimates into physics-informed embeddings. These embeddings regularize the egocentric rollouts toward physically valid dynamics. Experiments on the HOT3D dataset demonstrate consistent gains over strong baselines, and ablations validate the effectiveness of our physics-informed design. |
| title | Egocentric World Model for Photorealistic Hand-Object Interaction Synthesis |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2603.13615 |