CaptainCook4D: A Dataset for Understanding Errors in Procedural Activities
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929620175028224 |
|---|---|
| author | Peddi, Rohith Arya, Shivvrat Challa, Bharath Pallapothula, Likhitha Vyas, Akshay Gouripeddi, Bhavya Wang, Jikai Zhang, Qifan Komaragiri, Vasundhara Ragan, Eric Ruozzi, Nicholas Xiang, Yu Gogate, Vibhav |
| author_facet | Peddi, Rohith Arya, Shivvrat Challa, Bharath Pallapothula, Likhitha Vyas, Akshay Gouripeddi, Bhavya Wang, Jikai Zhang, Qifan Komaragiri, Vasundhara Ragan, Eric Ruozzi, Nicholas Xiang, Yu Gogate, Vibhav |
| contents | Following step-by-step procedures is an essential component of various activities carried out by individuals in their daily lives. These procedures serve as a guiding framework that helps to achieve goals efficiently, whether it is assembling furniture or preparing a recipe. However, the complexity and duration of procedural activities inherently increase the likelihood of making errors. Understanding such procedural activities from a sequence of frames is a challenging task that demands an accurate interpretation of visual information and the ability to reason about the structure of the activity. To this end, we collect a new egocentric 4D dataset, CaptainCook4D, comprising 384 recordings (94.5 hours) of people performing recipes in real kitchen environments. This dataset consists of two distinct types of activity: one in which participants adhere to the provided recipe instructions and another in which they deviate and induce errors. We provide 5.3K step annotations and 10K fine-grained action annotations and benchmark the dataset for the following tasks: supervised error recognition, multistep localization, and procedure learning |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_14556 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | CaptainCook4D: A Dataset for Understanding Errors in Procedural Activities Peddi, Rohith Arya, Shivvrat Challa, Bharath Pallapothula, Likhitha Vyas, Akshay Gouripeddi, Bhavya Wang, Jikai Zhang, Qifan Komaragiri, Vasundhara Ragan, Eric Ruozzi, Nicholas Xiang, Yu Gogate, Vibhav Computer Vision and Pattern Recognition Following step-by-step procedures is an essential component of various activities carried out by individuals in their daily lives. These procedures serve as a guiding framework that helps to achieve goals efficiently, whether it is assembling furniture or preparing a recipe. However, the complexity and duration of procedural activities inherently increase the likelihood of making errors. Understanding such procedural activities from a sequence of frames is a challenging task that demands an accurate interpretation of visual information and the ability to reason about the structure of the activity. To this end, we collect a new egocentric 4D dataset, CaptainCook4D, comprising 384 recordings (94.5 hours) of people performing recipes in real kitchen environments. This dataset consists of two distinct types of activity: one in which participants adhere to the provided recipe instructions and another in which they deviate and induce errors. We provide 5.3K step annotations and 10K fine-grained action annotations and benchmark the dataset for the following tasks: supervised error recognition, multistep localization, and procedure learning |
| title | CaptainCook4D: A Dataset for Understanding Errors in Procedural Activities |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.14556 |