Unsupervised Open-Vocabulary Object Localization in Videos
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911932881043456 |
|---|---|
| author | Fan, Ke Bai, Zechen Xiao, Tianjun Zietlow, Dominik Horn, Max Zhao, Zixu Simon-Gabriel, Carl-Johann Shou, Mike Zheng Locatello, Francesco Schiele, Bernt Brox, Thomas Zhang, Zheng Fu, Yanwei He, Tong |
| author_facet | Fan, Ke Bai, Zechen Xiao, Tianjun Zietlow, Dominik Horn, Max Zhao, Zixu Simon-Gabriel, Carl-Johann Shou, Mike Zheng Locatello, Francesco Schiele, Bernt Brox, Thomas Zhang, Zheng Fu, Yanwei He, Tong |
| contents | In this paper, we show that recent advances in video representation learning and pre-trained vision-language models allow for substantial improvements in self-supervised video object localization. We propose a method that first localizes objects in videos via an object-centric approach with slot attention and then assigns text to the obtained slots. The latter is achieved by an unsupervised way to read localized semantic information from the pre-trained CLIP model. The resulting video object localization is entirely unsupervised apart from the implicit annotation contained in CLIP, and it is effectively the first unsupervised approach that yields good results on regular video benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_09858 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Unsupervised Open-Vocabulary Object Localization in Videos Fan, Ke Bai, Zechen Xiao, Tianjun Zietlow, Dominik Horn, Max Zhao, Zixu Simon-Gabriel, Carl-Johann Shou, Mike Zheng Locatello, Francesco Schiele, Bernt Brox, Thomas Zhang, Zheng Fu, Yanwei He, Tong Computer Vision and Pattern Recognition In this paper, we show that recent advances in video representation learning and pre-trained vision-language models allow for substantial improvements in self-supervised video object localization. We propose a method that first localizes objects in videos via an object-centric approach with slot attention and then assigns text to the obtained slots. The latter is achieved by an unsupervised way to read localized semantic information from the pre-trained CLIP model. The resulting video object localization is entirely unsupervised apart from the implicit annotation contained in CLIP, and it is effectively the first unsupervised approach that yields good results on regular video benchmarks. |
| title | Unsupervised Open-Vocabulary Object Localization in Videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2309.09858 |