STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2019
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909400083464192 |
|---|---|
| author | Bhattacharya, Uttaran Mittal, Trisha Chandra, Rohan Randhavane, Tanmay Bera, Aniket Manocha, Dinesh |
| author_facet | Bhattacharya, Uttaran Mittal, Trisha Chandra, Rohan Randhavane, Tanmay Bera, Aniket Manocha, Dinesh |
| contents | We present a novel classifier network called STEP, to classify perceived human emotion from gaits, based on a Spatial Temporal Graph Convolutional Network (ST-GCN) architecture. Given an RGB video of an individual walking, our formulation implicitly exploits the gait features to classify the emotional state of the human into one of four emotions: happy, sad, angry, or neutral. We use hundreds of annotated real-world gait videos and augment them with thousands of annotated synthetic gaits generated using a novel generative network called STEP-Gen, built on an ST-GCN based Conditional Variational Autoencoder (CVAE). We incorporate a novel push-pull regularization loss in the CVAE formulation of STEP-Gen to generate realistic gaits and improve the classification accuracy of STEP. We also release a novel dataset (E-Gait), which consists of $2,177$ human gaits annotated with perceived emotions along with thousands of synthetic gaits. In practice, STEP can learn the affective features and exhibits classification accuracy of 89% on E-Gait, which is 14 - 30% more accurate over prior methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_1910_12906 |
| institution | arXiv |
| publishDate | 2019 |
| record_format | arxiv |
| spellingShingle | STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits Bhattacharya, Uttaran Mittal, Trisha Chandra, Rohan Randhavane, Tanmay Bera, Aniket Manocha, Dinesh Computer Vision and Pattern Recognition Robotics We present a novel classifier network called STEP, to classify perceived human emotion from gaits, based on a Spatial Temporal Graph Convolutional Network (ST-GCN) architecture. Given an RGB video of an individual walking, our formulation implicitly exploits the gait features to classify the emotional state of the human into one of four emotions: happy, sad, angry, or neutral. We use hundreds of annotated real-world gait videos and augment them with thousands of annotated synthetic gaits generated using a novel generative network called STEP-Gen, built on an ST-GCN based Conditional Variational Autoencoder (CVAE). We incorporate a novel push-pull regularization loss in the CVAE formulation of STEP-Gen to generate realistic gaits and improve the classification accuracy of STEP. We also release a novel dataset (E-Gait), which consists of $2,177$ human gaits annotated with perceived emotions along with thousands of synthetic gaits. In practice, STEP can learn the affective features and exhibits classification accuracy of 89% on E-Gait, which is 14 - 30% more accurate over prior methods. |
| title | STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/1910.12906 |