Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2021
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911760488857600 |
|---|---|
| author | Liang, Chen Wu, Yu Zhou, Tianfei Wang, Wenguan Yang, Zongxin Wei, Yunchao Yang, Yi |
| author_facet | Liang, Chen Wu, Yu Zhou, Tianfei Wang, Wenguan Yang, Zongxin Wei, Yunchao Yang, Yi |
| contents | Referring video object segmentation (RVOS) aims to segment video objects with the guidance of natural language reference. Previous methods typically tackle RVOS through directly grounding linguistic reference over the image lattice. Such bottom-up strategy fails to explore object-level cues, easily leading to inferior results. In this work, we instead put forward a two-stage, top-down RVOS solution. First, an exhaustive set of object tracklets is constructed by propagating object masks detected from several sampled frames to the entire video. Second, a Transformer-based tracklet-language grounding module is proposed, which models instance-level visual relations and cross-modal interactions simultaneously and efficiently. Our model ranks first place on CVPR2021 Referring Youtube-VOS challenge. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2106_01061 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation Liang, Chen Wu, Yu Zhou, Tianfei Wang, Wenguan Yang, Zongxin Wei, Yunchao Yang, Yi Computer Vision and Pattern Recognition Referring video object segmentation (RVOS) aims to segment video objects with the guidance of natural language reference. Previous methods typically tackle RVOS through directly grounding linguistic reference over the image lattice. Such bottom-up strategy fails to explore object-level cues, easily leading to inferior results. In this work, we instead put forward a two-stage, top-down RVOS solution. First, an exhaustive set of object tracklets is constructed by propagating object masks detected from several sampled frames to the entire video. Second, a Transformer-based tracklet-language grounding module is proposed, which models instance-level visual relations and cross-modal interactions simultaneously and efficiently. Our model ranks first place on CVPR2021 Referring Youtube-VOS challenge. |
| title | Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2106.01061 |