EgoInteract: Synthetic Egocentric Videos Generation for Interaction Understanding and Anticipation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917524483866624 |
|---|---|
| author | Leonardi, Rosario Ragusa, Francesco Materia, Daniele Passanisi, Alessandro Fort, James Engel, Jakob Farinella, Giovanni Maria |
| author_facet | Leonardi, Rosario Ragusa, Francesco Materia, Daniele Passanisi, Alessandro Fort, James Engel, Jakob Farinella, Giovanni Maria |
| contents | Collecting large-scale egocentric video datasets with dense spatial and temporal annotations is costly, slow, and often constrained by environmental biases, privacy constraints, and limited coverage of interaction patterns. While synthetic data has shown strong potential in several vision domains, its use for egocentric perception remains relatively underexplored, especially for tasks requiring temporally coherent human-object interactions. In this work, we introduce EgoInteract, a controllable simulator for egocentric video generation designed to model fine-grained egocentric interactions and their temporal dynamics. The simulator enables precise control over camera, human body and hand motion, object manipulation, and scene composition across diverse environments. Building on this framework, we generate a synthetic egocentric video dataset with dense spatial and temporal annotations for temporal action segmentation, next-active object detection, interaction anticipation, and hand-object interaction detection. We evaluate models trained with simulated data on multiple real-world egocentric benchmarks spanning diverse environments, object categories, and interaction patterns. Results show consistent improvements over strong baselines across tasks and datasets, demonstrating the effectiveness and transferability of our simulation-based approach. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_18214 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | EgoInteract: Synthetic Egocentric Videos Generation for Interaction Understanding and Anticipation Leonardi, Rosario Ragusa, Francesco Materia, Daniele Passanisi, Alessandro Fort, James Engel, Jakob Farinella, Giovanni Maria Computer Vision and Pattern Recognition Collecting large-scale egocentric video datasets with dense spatial and temporal annotations is costly, slow, and often constrained by environmental biases, privacy constraints, and limited coverage of interaction patterns. While synthetic data has shown strong potential in several vision domains, its use for egocentric perception remains relatively underexplored, especially for tasks requiring temporally coherent human-object interactions. In this work, we introduce EgoInteract, a controllable simulator for egocentric video generation designed to model fine-grained egocentric interactions and their temporal dynamics. The simulator enables precise control over camera, human body and hand motion, object manipulation, and scene composition across diverse environments. Building on this framework, we generate a synthetic egocentric video dataset with dense spatial and temporal annotations for temporal action segmentation, next-active object detection, interaction anticipation, and hand-object interaction detection. We evaluate models trained with simulated data on multiple real-world egocentric benchmarks spanning diverse environments, object categories, and interaction patterns. Results show consistent improvements over strong baselines across tasks and datasets, demonstrating the effectiveness and transferability of our simulation-based approach. |
| title | EgoInteract: Synthetic Egocentric Videos Generation for Interaction Understanding and Anticipation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.18214 |