Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.20270 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909368739430400 |
|---|---|
| author | Ali, Abid Dai, Rui Marisetty, Ashish Astruc, Guillaume Thonnat, Monique Odobez, Jean-Marc Thümmler, Susanne Bremond, Francois |
| author_facet | Ali, Abid Dai, Rui Marisetty, Ashish Astruc, Guillaume Thonnat, Monique Odobez, Jean-Marc Thümmler, Susanne Bremond, Francois |
| contents | The computer vision community has explored dyadic interactions for atomic actions such as pushing, carrying-object, etc. However, with the advancement in deep learning models, there is a need to explore more complex dyadic situations such as loose interactions. These are interactions where two people perform certain atomic activities to complete a global action irrespective of temporal synchronisation and physical engagement, like cooking-together for example. Analysing these types of dyadic-interactions has several useful applications in the medical domain for social-skills development and mental health diagnosis.
To achieve this, we propose a novel dual-path architecture to capture the loose interaction between two individuals. Our model learns global abstract features from each stream via a CNNs backbone and fuses them using a new Global-Layer-Attention module based on a cross-attention strategy. We evaluate our model on real-world autism diagnoses such as our Loose-Interaction dataset, and the publicly available Autism dataset for loose interactions. Our network achieves baseline results on the Loose-Interaction and SOTA results on the Autism datasets. Moreover, we study different social interactions by experimenting on a publicly available dataset i.e. NTU-RGB+D (interactive classes from both NTU-60 and NTU-120). We have found that different interactions require different network designs. We also compare a slightly different version of our method by incorporating time information to address tight interactions achieving SOTA results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_20270 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Loose Social-Interaction Recognition in Real-world Therapy Scenarios Ali, Abid Dai, Rui Marisetty, Ashish Astruc, Guillaume Thonnat, Monique Odobez, Jean-Marc Thümmler, Susanne Bremond, Francois Computer Vision and Pattern Recognition The computer vision community has explored dyadic interactions for atomic actions such as pushing, carrying-object, etc. However, with the advancement in deep learning models, there is a need to explore more complex dyadic situations such as loose interactions. These are interactions where two people perform certain atomic activities to complete a global action irrespective of temporal synchronisation and physical engagement, like cooking-together for example. Analysing these types of dyadic-interactions has several useful applications in the medical domain for social-skills development and mental health diagnosis. To achieve this, we propose a novel dual-path architecture to capture the loose interaction between two individuals. Our model learns global abstract features from each stream via a CNNs backbone and fuses them using a new Global-Layer-Attention module based on a cross-attention strategy. We evaluate our model on real-world autism diagnoses such as our Loose-Interaction dataset, and the publicly available Autism dataset for loose interactions. Our network achieves baseline results on the Loose-Interaction and SOTA results on the Autism datasets. Moreover, we study different social interactions by experimenting on a publicly available dataset i.e. NTU-RGB+D (interactive classes from both NTU-60 and NTU-120). We have found that different interactions require different network designs. We also compare a slightly different version of our method by incorporating time information to address tight interactions achieving SOTA results. |
| title | Loose Social-Interaction Recognition in Real-world Therapy Scenarios |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.20270 |