Are you Struggling? Dataset and Baselines for Struggle Determination in Assembly Videos
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912536099553280 |
|---|---|
| author | Feng, Shijia Wray, Michael Sullivan, Brian Jang, Youngkyoon Ludwig, Casimir Gilchrist, Iain Mayol-Cuevas, Walterio |
| author_facet | Feng, Shijia Wray, Michael Sullivan, Brian Jang, Youngkyoon Ludwig, Casimir Gilchrist, Iain Mayol-Cuevas, Walterio |
| contents | Determining when people are struggling allows for a finer-grained understanding of actions that complements conventional action classification and error detection. Struggle detection, as defined in this paper, is a distinct and important task that can be identified without explicit step or activity knowledge. We introduce the first struggle dataset with three real-world problem-solving activities that are labelled by both expert and crowd-source annotators. Video segments were scored w.r.t. their level of struggle using a forced choice 4-point scale. This dataset contains 5.1 hours of video from 73 participants. We conducted a series of experiments to identify the most suitable modelling approaches for struggle determination. Additionally, we compared various deep learning models, establishing baseline results for struggle classification, struggle regression, and struggle label distribution learning. Our results indicate that struggle detection in video can achieve up to $88.24\%$ accuracy in binary classification, while detecting the level of struggle in a four-way classification setting performs lower, with an overall accuracy of $52.45\%$. Our work is motivated toward a more comprehensive understanding of action in video and potentially the improvement of assistive systems that analyse struggle and can better support users during manual activities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_11057 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Are you Struggling? Dataset and Baselines for Struggle Determination in Assembly Videos Feng, Shijia Wray, Michael Sullivan, Brian Jang, Youngkyoon Ludwig, Casimir Gilchrist, Iain Mayol-Cuevas, Walterio Computer Vision and Pattern Recognition Determining when people are struggling allows for a finer-grained understanding of actions that complements conventional action classification and error detection. Struggle detection, as defined in this paper, is a distinct and important task that can be identified without explicit step or activity knowledge. We introduce the first struggle dataset with three real-world problem-solving activities that are labelled by both expert and crowd-source annotators. Video segments were scored w.r.t. their level of struggle using a forced choice 4-point scale. This dataset contains 5.1 hours of video from 73 participants. We conducted a series of experiments to identify the most suitable modelling approaches for struggle determination. Additionally, we compared various deep learning models, establishing baseline results for struggle classification, struggle regression, and struggle label distribution learning. Our results indicate that struggle detection in video can achieve up to $88.24\%$ accuracy in binary classification, while detecting the level of struggle in a four-way classification setting performs lower, with an overall accuracy of $52.45\%$. Our work is motivated toward a more comprehensive understanding of action in video and potentially the improvement of assistive systems that analyse struggle and can better support users during manual activities. |
| title | Are you Struggling? Dataset and Baselines for Struggle Determination in Assembly Videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.11057 |