Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2021
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2110.13385 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929288071086080 |
|---|---|
| author | Wang, Qingtian Peng, Jianlin Shi, Shuze Liu, Tingxi He, Jiabin Weng, Renliang |
| author_facet | Wang, Qingtian Peng, Jianlin Shi, Shuze Liu, Tingxi He, Jiabin Weng, Renliang |
| contents | Recently, Transformer-based networks have shown great promise on skeleton-based action recognition tasks. The ability to capture global and local dependencies is the key to success while it also brings quadratic computation and memory cost. Another problem is that previous studies mainly focus on the relationships among individual joints, which often suffers from the noisy skeleton joints introduced by the noisy inputs of sensors or inaccurate estimations. To address the above issues, we propose a novel Transformer-based network (IIP-Transformer). Instead of exploiting interactions among individual joints, our IIP-Transformer incorporates body joints and parts interactions simultaneously and thus can capture both joint-level (intra-part) and part-level (inter-part) dependencies efficiently and effectively. From the data aspect, we introduce a part-level skeleton data encoding that significantly reduces the computational complexity and is more robust to joint-level skeleton noise. Besides, a new part-level data augmentation is proposed to improve the performance of the model. On two large-scale datasets, NTU-RGB+D 60 and NTU RGB+D 120, the proposed IIP-Transformer achieves the-state-of-art performance with more than 8x less computational complexity than DSTA-Net, which is the SOTA Transformer-based method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2110_13385 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | IIP-Transformer: Intra-Inter-Part Transformer for Skeleton-Based Action Recognition Wang, Qingtian Peng, Jianlin Shi, Shuze Liu, Tingxi He, Jiabin Weng, Renliang Computer Vision and Pattern Recognition Recently, Transformer-based networks have shown great promise on skeleton-based action recognition tasks. The ability to capture global and local dependencies is the key to success while it also brings quadratic computation and memory cost. Another problem is that previous studies mainly focus on the relationships among individual joints, which often suffers from the noisy skeleton joints introduced by the noisy inputs of sensors or inaccurate estimations. To address the above issues, we propose a novel Transformer-based network (IIP-Transformer). Instead of exploiting interactions among individual joints, our IIP-Transformer incorporates body joints and parts interactions simultaneously and thus can capture both joint-level (intra-part) and part-level (inter-part) dependencies efficiently and effectively. From the data aspect, we introduce a part-level skeleton data encoding that significantly reduces the computational complexity and is more robust to joint-level skeleton noise. Besides, a new part-level data augmentation is proposed to improve the performance of the model. On two large-scale datasets, NTU-RGB+D 60 and NTU RGB+D 120, the proposed IIP-Transformer achieves the-state-of-art performance with more than 8x less computational complexity than DSTA-Net, which is the SOTA Transformer-based method. |
| title | IIP-Transformer: Intra-Inter-Part Transformer for Skeleton-Based Action Recognition |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2110.13385 |