SignMAE: Segmentation-Driven Self-Supervised Learning for Sign Language Recognition
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915976607432704 |
|---|---|
| author | Xie, Kunyuan Cai, Zhixi Stefanov, Kalin |
| author_facet | Xie, Kunyuan Cai, Zhixi Stefanov, Kalin |
| contents | Subtle hand differences make sign language recognition challenging, yet many existing methods rely on encoders pretrained on generic action datasets that poorly capture such fine-grained cues. We propose a self-supervised pretraining method for sign language recognition that uses segmentation-based masking to adapt to the presence and motion of key body parts, rather than treating hand poses as static visual tokens. The resulting mask-and-reconstruct objective improves fine-grained sign representation learning. On WLASL, NMFs-CSL, and Slovo, our encoder achieves state-of-the-art performance, improving per-instance and per-class Top-1 accuracy while using fewer input frames and modalities than comparable encoders. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_02094 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SignMAE: Segmentation-Driven Self-Supervised Learning for Sign Language Recognition Xie, Kunyuan Cai, Zhixi Stefanov, Kalin Computer Vision and Pattern Recognition Subtle hand differences make sign language recognition challenging, yet many existing methods rely on encoders pretrained on generic action datasets that poorly capture such fine-grained cues. We propose a self-supervised pretraining method for sign language recognition that uses segmentation-based masking to adapt to the presence and motion of key body parts, rather than treating hand poses as static visual tokens. The resulting mask-and-reconstruct objective improves fine-grained sign representation learning. On WLASL, NMFs-CSL, and Slovo, our encoder achieves state-of-the-art performance, improving per-instance and per-class Top-1 accuracy while using fewer input frames and modalities than comparable encoders. |
| title | SignMAE: Segmentation-Driven Self-Supervised Learning for Sign Language Recognition |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.02094 |