Learning Cross-View Object Correspondence via Cycle-Consistent Mask Prediction
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911542423846912 |
|---|---|
| author | Yan, Shannan Zheng, Leqi Lv, Keyu Ni, Jingchen Wei, Hongyang Zhang, Jiajun Wang, Guangting Lyu, Jing Yuan, Chun Rao, Fengyun |
| author_facet | Yan, Shannan Zheng, Leqi Lv, Keyu Ni, Jingchen Wei, Hongyang Zhang, Jiajun Wang, Guangting Lyu, Jing Yuan, Chun Rao, Fengyun |
| contents | We study the task of establishing object-level visual correspondence across different viewpoints in videos, focusing on the challenging egocentric-to-exocentric and exocentric-to-egocentric scenarios. We propose a simple yet effective framework based on conditional binary segmentation, where an object query mask is encoded into a latent representation to guide the localization of the corresponding object in a target video. To encourage robust, view-invariant representations, we introduce a cycle-consistency training objective: the predicted mask in the target view is projected back to the source view to reconstruct the original query mask. This bidirectional constraint provides a strong self-supervisory signal without requiring ground-truth annotations and enables test-time training (TTT) at inference. Experiments on the Ego-Exo4D and HANDAL-X benchmarks demonstrate the effectiveness of our optimization objective and TTT strategy, achieving state-of-the-art performance. The code is available at https://github.com/shannany0606/CCMP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_18996 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Learning Cross-View Object Correspondence via Cycle-Consistent Mask Prediction Yan, Shannan Zheng, Leqi Lv, Keyu Ni, Jingchen Wei, Hongyang Zhang, Jiajun Wang, Guangting Lyu, Jing Yuan, Chun Rao, Fengyun Computer Vision and Pattern Recognition We study the task of establishing object-level visual correspondence across different viewpoints in videos, focusing on the challenging egocentric-to-exocentric and exocentric-to-egocentric scenarios. We propose a simple yet effective framework based on conditional binary segmentation, where an object query mask is encoded into a latent representation to guide the localization of the corresponding object in a target video. To encourage robust, view-invariant representations, we introduce a cycle-consistency training objective: the predicted mask in the target view is projected back to the source view to reconstruct the original query mask. This bidirectional constraint provides a strong self-supervisory signal without requiring ground-truth annotations and enables test-time training (TTT) at inference. Experiments on the Ego-Exo4D and HANDAL-X benchmarks demonstrate the effectiveness of our optimization objective and TTT strategy, achieving state-of-the-art performance. The code is available at https://github.com/shannany0606/CCMP. |
| title | Learning Cross-View Object Correspondence via Cycle-Consistent Mask Prediction |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2602.18996 |