The Instance-centric Transformer for the RVOS Track of LSVOS Challenge: 3rd Place Solution
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914917672550400 |
|---|---|
| author | Cao, Bin Zhang, Yisi Wang, Hanyi He, Xingjian Liu, Jing |
| author_facet | Cao, Bin Zhang, Yisi Wang, Hanyi He, Xingjian Liu, Jing |
| contents | Referring Video Object Segmentation is an emerging multi-modal task that aims to segment objects in the video given a natural language expression. In this work, we build two instance-centric models and fuse predicted results from frame-level and instance-level. First, we introduce instance mask into the DETR-based model for query initialization to achieve temporal enhancement and employ SAM for spatial refinement. Secondly, we build an instance retrieval model conducting binary instance mask classification whether the instance is referred. Finally, we fuse predicted results and our method achieved a score of 52.67 J&F in the validation phase and 60.36 J&F in the test phase, securing the final ranking of 3rd place in the 6-th LSVOS Challenge RVOS Track. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_10541 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Instance-centric Transformer for the RVOS Track of LSVOS Challenge: 3rd Place Solution Cao, Bin Zhang, Yisi Wang, Hanyi He, Xingjian Liu, Jing Computer Vision and Pattern Recognition Referring Video Object Segmentation is an emerging multi-modal task that aims to segment objects in the video given a natural language expression. In this work, we build two instance-centric models and fuse predicted results from frame-level and instance-level. First, we introduce instance mask into the DETR-based model for query initialization to achieve temporal enhancement and employ SAM for spatial refinement. Secondly, we build an instance retrieval model conducting binary instance mask classification whether the instance is referred. Finally, we fuse predicted results and our method achieved a score of 52.67 J&F in the validation phase and 60.36 J&F in the test phase, securing the final ranking of 3rd place in the 6-th LSVOS Challenge RVOS Track. |
| title | The Instance-centric Transformer for the RVOS Track of LSVOS Challenge: 3rd Place Solution |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.10541 |