Text-Enhanced Zero-Shot Action Recognition: A training-free approach
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912022428385280 |
|---|---|
| author | Bosetti, Massimo Zhang, Shibingfeng Liberatori, Benedetta Zara, Giacomo Ricci, Elisa Rota, Paolo |
| author_facet | Bosetti, Massimo Zhang, Shibingfeng Liberatori, Benedetta Zara, Giacomo Ricci, Elisa Rota, Paolo |
| contents | Vision-language models (VLMs) have demonstrated remarkable performance across various visual tasks, leveraging joint learning of visual and textual representations. While these models excel in zero-shot image tasks, their application to zero-shot video action recognition (ZSVAR) remains challenging due to the dynamic and temporal nature of actions. Existing methods for ZS-VAR typically require extensive training on specific datasets, which can be resource-intensive and may introduce domain biases. In this work, we propose Text-Enhanced Action Recognition (TEAR), a simple approach to ZS-VAR that is training-free and does not require the availability of training data or extensive computational resources. Drawing inspiration from recent findings in vision and language literature, we utilize action descriptors for decomposition and contextual information to enhance zero-shot action recognition. Through experiments on UCF101, HMDB51, and Kinetics-600 datasets, we showcase the effectiveness and applicability of our proposed approach in addressing the challenges of ZS-VAR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_16412 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Text-Enhanced Zero-Shot Action Recognition: A training-free approach Bosetti, Massimo Zhang, Shibingfeng Liberatori, Benedetta Zara, Giacomo Ricci, Elisa Rota, Paolo Computer Vision and Pattern Recognition Vision-language models (VLMs) have demonstrated remarkable performance across various visual tasks, leveraging joint learning of visual and textual representations. While these models excel in zero-shot image tasks, their application to zero-shot video action recognition (ZSVAR) remains challenging due to the dynamic and temporal nature of actions. Existing methods for ZS-VAR typically require extensive training on specific datasets, which can be resource-intensive and may introduce domain biases. In this work, we propose Text-Enhanced Action Recognition (TEAR), a simple approach to ZS-VAR that is training-free and does not require the availability of training data or extensive computational resources. Drawing inspiration from recent findings in vision and language literature, we utilize action descriptors for decomposition and contextual information to enhance zero-shot action recognition. Through experiments on UCF101, HMDB51, and Kinetics-600 datasets, we showcase the effectiveness and applicability of our proposed approach in addressing the challenges of ZS-VAR. |
| title | Text-Enhanced Zero-Shot Action Recognition: A training-free approach |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.16412 |