SocialGesture: Delving into Multi-person Gesture Understanding
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913774886191104 |
|---|---|
| author | Cao, Xu Virupaksha, Pranav Jia, Wenqi Lai, Bolin Ryan, Fiona Lee, Sangmin Rehg, James M. |
| author_facet | Cao, Xu Virupaksha, Pranav Jia, Wenqi Lai, Bolin Ryan, Fiona Lee, Sangmin Rehg, James M. |
| contents | Previous research in human gesture recognition has largely overlooked multi-person interactions, which are crucial for understanding the social context of naturally occurring gestures. This limitation in existing datasets presents a significant challenge in aligning human gestures with other modalities like language and speech. To address this issue, we introduce SocialGesture, the first large-scale dataset specifically designed for multi-person gesture analysis. SocialGesture features a diverse range of natural scenarios and supports multiple gesture analysis tasks, including video-based recognition and temporal localization, providing a valuable resource for advancing the study of gesture during complex social interactions. Furthermore, we propose a novel visual question answering (VQA) task to benchmark vision language models'(VLMs) performance on social gesture understanding. Our findings highlight several limitations of current gesture recognition models, offering insights into future directions for improvement in this field. SocialGesture is available at huggingface.co/datasets/IrohXu/SocialGesture. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_02244 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SocialGesture: Delving into Multi-person Gesture Understanding Cao, Xu Virupaksha, Pranav Jia, Wenqi Lai, Bolin Ryan, Fiona Lee, Sangmin Rehg, James M. Computer Vision and Pattern Recognition Previous research in human gesture recognition has largely overlooked multi-person interactions, which are crucial for understanding the social context of naturally occurring gestures. This limitation in existing datasets presents a significant challenge in aligning human gestures with other modalities like language and speech. To address this issue, we introduce SocialGesture, the first large-scale dataset specifically designed for multi-person gesture analysis. SocialGesture features a diverse range of natural scenarios and supports multiple gesture analysis tasks, including video-based recognition and temporal localization, providing a valuable resource for advancing the study of gesture during complex social interactions. Furthermore, we propose a novel visual question answering (VQA) task to benchmark vision language models'(VLMs) performance on social gesture understanding. Our findings highlight several limitations of current gesture recognition models, offering insights into future directions for improvement in this field. SocialGesture is available at huggingface.co/datasets/IrohXu/SocialGesture. |
| title | SocialGesture: Delving into Multi-person Gesture Understanding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.02244 |