Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.15668 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910686328651776 |
|---|---|
| author | Martins, Gonçalo Vinagre Magalhães, João Quinaz, Afonso Viegas, Carla Cavaco, Sofia |
| author_facet | Martins, Gonçalo Vinagre Magalhães, João Quinaz, Afonso Viegas, Carla Cavaco, Sofia |
| contents | SLVideo is a video moment retrieval system for Sign Language videos that incorporates facial expressions, addressing this gap in existing technology. The system extracts embedding representations for the hand and face signs from video frames to capture the signs in their entirety, enabling users to search for a specific sign language video segment with text queries. A collection of eight hours of annotated Portuguese Sign Language videos is used as the dataset, and a CLIP model is used to generate the embeddings. The initial results are promising in a zero-shot setting. In addition, SLVideo incorporates a thesaurus that enables users to search for similar signs to those retrieved, using the video segment embeddings, and also supports the edition and creation of video sign language annotations. Project web page: https://novasearch.github.io/SLVideo/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_15668 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SLVideo: A Sign Language Video Moment Retrieval Framework Martins, Gonçalo Vinagre Magalhães, João Quinaz, Afonso Viegas, Carla Cavaco, Sofia Computer Vision and Pattern Recognition Artificial Intelligence SLVideo is a video moment retrieval system for Sign Language videos that incorporates facial expressions, addressing this gap in existing technology. The system extracts embedding representations for the hand and face signs from video frames to capture the signs in their entirety, enabling users to search for a specific sign language video segment with text queries. A collection of eight hours of annotated Portuguese Sign Language videos is used as the dataset, and a CLIP model is used to generate the embeddings. The initial results are promising in a zero-shot setting. In addition, SLVideo incorporates a thesaurus that enables users to search for similar signs to those retrieved, using the video segment embeddings, and also supports the edition and creation of video sign language annotations. Project web page: https://novasearch.github.io/SLVideo/ |
| title | SLVideo: A Sign Language Video Moment Retrieval Framework |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2407.15668 |