O-MaMa: Learning Object Mask Matching between Egocentric and Exocentric Views
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909805590872064 |
|---|---|
| author | Mur-Labadia, Lorenzo Santos-Villafranca, Maria Bermudez-Cameo, Jesus Perez-Yus, Alejandro Martinez-Cantin, Ruben Guerrero, Jose J. |
| author_facet | Mur-Labadia, Lorenzo Santos-Villafranca, Maria Bermudez-Cameo, Jesus Perez-Yus, Alejandro Martinez-Cantin, Ruben Guerrero, Jose J. |
| contents | Understanding the world from multiple perspectives is essential for intelligent systems operating together, where segmenting common objects across different views remains an open problem. We introduce a new approach that re-defines cross-image segmentation by treating it as a mask matching task. Our method consists of: (1) A Mask-Context Encoder that pools dense DINOv2 semantic features to obtain discriminative object-level representations from FastSAM mask candidates, (2) an Ego$\leftrightarrow$Exo Cross-Attention that fuses multi-perspective observations, (3) a Mask Matching contrastive loss that aligns cross-view features in a shared latent space, and (4) a Hard Negative Adjacent Mining strategy to encourage the model to better differentiate between nearby objects. O-MaMa achieves the state of the art in the Ego-Exo4D Correspondences benchmark, obtaining relative gains of +22% and +76% in the Ego2Exo and Exo2Ego IoU against the official challenge baselines, and a +13% and +6% compared with the SOTA with 1% of the training parameters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_06026 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | O-MaMa: Learning Object Mask Matching between Egocentric and Exocentric Views Mur-Labadia, Lorenzo Santos-Villafranca, Maria Bermudez-Cameo, Jesus Perez-Yus, Alejandro Martinez-Cantin, Ruben Guerrero, Jose J. Computer Vision and Pattern Recognition Understanding the world from multiple perspectives is essential for intelligent systems operating together, where segmenting common objects across different views remains an open problem. We introduce a new approach that re-defines cross-image segmentation by treating it as a mask matching task. Our method consists of: (1) A Mask-Context Encoder that pools dense DINOv2 semantic features to obtain discriminative object-level representations from FastSAM mask candidates, (2) an Ego$\leftrightarrow$Exo Cross-Attention that fuses multi-perspective observations, (3) a Mask Matching contrastive loss that aligns cross-view features in a shared latent space, and (4) a Hard Negative Adjacent Mining strategy to encourage the model to better differentiate between nearby objects. O-MaMa achieves the state of the art in the Ego-Exo4D Correspondences benchmark, obtaining relative gains of +22% and +76% in the Ego2Exo and Exo2Ego IoU against the official challenge baselines, and a +13% and +6% compared with the SOTA with 1% of the training parameters. |
| title | O-MaMa: Learning Object Mask Matching between Egocentric and Exocentric Views |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.06026 |