DIV-FF: Dynamic Image-Video Feature Fields For Environment Understanding in Egocentric Videos
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915191339352064 |
|---|---|
| author | Mur-Labadia, Lorenzo Guerrero, Josechu Martinez-Cantin, Ruben |
| author_facet | Mur-Labadia, Lorenzo Guerrero, Josechu Martinez-Cantin, Ruben |
| contents | Environment understanding in egocentric videos is an important step for applications like robotics, augmented reality and assistive technologies. These videos are characterized by dynamic interactions and a strong dependence on the wearer engagement with the environment. Traditional approaches often focus on isolated clips or fail to integrate rich semantic and geometric information, limiting scene comprehension. We introduce Dynamic Image-Video Feature Fields (DIV FF), a framework that decomposes the egocentric scene into persistent, dynamic, and actor based components while integrating both image and video language features. Our model enables detailed segmentation, captures affordances, understands the surroundings and maintains consistent understanding over time. DIV-FF outperforms state-of-the-art methods, particularly in dynamically evolving scenarios, demonstrating its potential to advance long term, spatio temporal scene understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_08344 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DIV-FF: Dynamic Image-Video Feature Fields For Environment Understanding in Egocentric Videos Mur-Labadia, Lorenzo Guerrero, Josechu Martinez-Cantin, Ruben Computer Vision and Pattern Recognition Environment understanding in egocentric videos is an important step for applications like robotics, augmented reality and assistive technologies. These videos are characterized by dynamic interactions and a strong dependence on the wearer engagement with the environment. Traditional approaches often focus on isolated clips or fail to integrate rich semantic and geometric information, limiting scene comprehension. We introduce Dynamic Image-Video Feature Fields (DIV FF), a framework that decomposes the egocentric scene into persistent, dynamic, and actor based components while integrating both image and video language features. Our model enables detailed segmentation, captures affordances, understands the surroundings and maintains consistent understanding over time. DIV-FF outperforms state-of-the-art methods, particularly in dynamically evolving scenarios, demonstrating its potential to advance long term, spatio temporal scene understanding. |
| title | DIV-FF: Dynamic Image-Video Feature Fields For Environment Understanding in Egocentric Videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.08344 |