Real-Time Human Action Recognition on Embedded Platforms
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909761596817408 |
|---|---|
| author | Wang, Ruiqi Wang, Zichen Gao, Peiqi Li, Mingzhen Jeong, Jaehwan Xu, Yihang Lee, Yejin Baum, Carolyn M. Connor, Lisa Tabor Lu, Chenyang |
| author_facet | Wang, Ruiqi Wang, Zichen Gao, Peiqi Li, Mingzhen Jeong, Jaehwan Xu, Yihang Lee, Yejin Baum, Carolyn M. Connor, Lisa Tabor Lu, Chenyang |
| contents | With advancements in computer vision and deep learning, video-based human action recognition (HAR) has become practical. However, due to the complexity of the computation pipeline, running HAR on live video streams incurs excessive delays on embedded platforms. This work tackles the real-time performance challenges of HAR with four contributions: 1) an experimental study identifying a standard Optical Flow (OF) extraction technique as the latency bottleneck in a state-of-the-art HAR pipeline, 2) an exploration of the latency-accuracy tradeoff between the standard and deep learning approaches to OF extraction, which highlights the need for a novel, efficient motion feature extractor, 3) the design of Integrated Motion Feature Extractor (IMFE), a novel single-shot neural network architecture for motion feature extraction with drastic improvement in latency, 4) the development of RT-HARE, a real-time HAR system tailored for embedded platforms. Experimental results on an Nvidia Jetson Xavier NX platform demonstrated that RT-HARE realizes real-time HAR at a video frame rate of 30 frames per second while delivering high levels of recognition accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_05662 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Real-Time Human Action Recognition on Embedded Platforms Wang, Ruiqi Wang, Zichen Gao, Peiqi Li, Mingzhen Jeong, Jaehwan Xu, Yihang Lee, Yejin Baum, Carolyn M. Connor, Lisa Tabor Lu, Chenyang Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning With advancements in computer vision and deep learning, video-based human action recognition (HAR) has become practical. However, due to the complexity of the computation pipeline, running HAR on live video streams incurs excessive delays on embedded platforms. This work tackles the real-time performance challenges of HAR with four contributions: 1) an experimental study identifying a standard Optical Flow (OF) extraction technique as the latency bottleneck in a state-of-the-art HAR pipeline, 2) an exploration of the latency-accuracy tradeoff between the standard and deep learning approaches to OF extraction, which highlights the need for a novel, efficient motion feature extractor, 3) the design of Integrated Motion Feature Extractor (IMFE), a novel single-shot neural network architecture for motion feature extraction with drastic improvement in latency, 4) the development of RT-HARE, a real-time HAR system tailored for embedded platforms. Experimental results on an Nvidia Jetson Xavier NX platform demonstrated that RT-HARE realizes real-time HAR at a video frame rate of 30 frames per second while delivering high levels of recognition accuracy. |
| title | Real-Time Human Action Recognition on Embedded Platforms |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2409.05662 |