Exploring Self-supervised Skeleton-based Action Recognition in Occluded Environments
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915244248399872 |
|---|---|
| author | Chen, Yifei Peng, Kunyu Roitberg, Alina Schneider, David Zhang, Jiaming Zheng, Junwei Chen, Yufan Liu, Ruiping Yang, Kailun Stiefelhagen, Rainer |
| author_facet | Chen, Yifei Peng, Kunyu Roitberg, Alina Schneider, David Zhang, Jiaming Zheng, Junwei Chen, Yufan Liu, Ruiping Yang, Kailun Stiefelhagen, Rainer |
| contents | To integrate action recognition into autonomous robotic systems, it is essential to address challenges such as person occlusions-a common yet often overlooked scenario in existing self-supervised skeleton-based action recognition methods. In this work, we propose IosPSTL, a simple and effective self-supervised learning framework designed to handle occlusions. IosPSTL combines a cluster-agnostic KNN imputer with an Occluded Partial Spatio-Temporal Learning (OPSTL) strategy. First, we pre-train the model on occluded skeleton sequences. Then, we introduce a cluster-agnostic KNN imputer that performs semantic grouping using k-means clustering on sequence embeddings. It imputes missing skeleton data by applying K-Nearest Neighbors in the latent space, leveraging nearby sample representations to restore occluded joints. This imputation generates more complete skeleton sequences, which significantly benefits downstream self-supervised models. To further enhance learning, the OPSTL module incorporates Adaptive Spatial Masking (ASM) to make better use of intact, high-quality skeleton sequences during training. Our method achieves state-of-the-art performance on the occluded versions of the NTU-60 and NTU-120 datasets, demonstrating its robustness and effectiveness under challenging conditions. Code is available at https://github.com/cyfml/OPSTL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_12029 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Exploring Self-supervised Skeleton-based Action Recognition in Occluded Environments Chen, Yifei Peng, Kunyu Roitberg, Alina Schneider, David Zhang, Jiaming Zheng, Junwei Chen, Yufan Liu, Ruiping Yang, Kailun Stiefelhagen, Rainer Computer Vision and Pattern Recognition Multimedia Robotics Image and Video Processing To integrate action recognition into autonomous robotic systems, it is essential to address challenges such as person occlusions-a common yet often overlooked scenario in existing self-supervised skeleton-based action recognition methods. In this work, we propose IosPSTL, a simple and effective self-supervised learning framework designed to handle occlusions. IosPSTL combines a cluster-agnostic KNN imputer with an Occluded Partial Spatio-Temporal Learning (OPSTL) strategy. First, we pre-train the model on occluded skeleton sequences. Then, we introduce a cluster-agnostic KNN imputer that performs semantic grouping using k-means clustering on sequence embeddings. It imputes missing skeleton data by applying K-Nearest Neighbors in the latent space, leveraging nearby sample representations to restore occluded joints. This imputation generates more complete skeleton sequences, which significantly benefits downstream self-supervised models. To further enhance learning, the OPSTL module incorporates Adaptive Spatial Masking (ASM) to make better use of intact, high-quality skeleton sequences during training. Our method achieves state-of-the-art performance on the occluded versions of the NTU-60 and NTU-120 datasets, demonstrating its robustness and effectiveness under challenging conditions. Code is available at https://github.com/cyfml/OPSTL. |
| title | Exploring Self-supervised Skeleton-based Action Recognition in Occluded Environments |
| topic | Computer Vision and Pattern Recognition Multimedia Robotics Image and Video Processing |
| url | https://arxiv.org/abs/2309.12029 |