PREGO: online mistake detection in PRocedural EGOcentric videos
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909206167158784 |
|---|---|
| author | Flaborea, Alessandro di Melendugno, Guido Maria D'Amely Plini, Leonardo Scofano, Luca De Matteis, Edoardo Furnari, Antonino Farinella, Giovanni Maria Galasso, Fabio |
| author_facet | Flaborea, Alessandro di Melendugno, Guido Maria D'Amely Plini, Leonardo Scofano, Luca De Matteis, Edoardo Furnari, Antonino Farinella, Giovanni Maria Galasso, Fabio |
| contents | Promptly identifying procedural errors from egocentric videos in an online setting is highly challenging and valuable for detecting mistakes as soon as they happen. This capability has a wide range of applications across various fields, such as manufacturing and healthcare. The nature of procedural mistakes is open-set since novel types of failures might occur, which calls for one-class classifiers trained on correctly executed procedures. However, no technique can currently detect open-set procedural mistakes online. We propose PREGO, the first online one-class classification model for mistake detection in PRocedural EGOcentric videos. PREGO is based on an online action recognition component to model the current action, and a symbolic reasoning module to predict the next actions. Mistake detection is performed by comparing the recognized current action with the expected future one. We evaluate PREGO on two procedural egocentric video datasets, Assembly101 and Epic-tent, which we adapt for online benchmarking of procedural mistake detection to establish suitable benchmarks, thus defining the Assembly101-O and Epic-tent-O datasets, respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_01933 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PREGO: online mistake detection in PRocedural EGOcentric videos Flaborea, Alessandro di Melendugno, Guido Maria D'Amely Plini, Leonardo Scofano, Luca De Matteis, Edoardo Furnari, Antonino Farinella, Giovanni Maria Galasso, Fabio Computer Vision and Pattern Recognition Promptly identifying procedural errors from egocentric videos in an online setting is highly challenging and valuable for detecting mistakes as soon as they happen. This capability has a wide range of applications across various fields, such as manufacturing and healthcare. The nature of procedural mistakes is open-set since novel types of failures might occur, which calls for one-class classifiers trained on correctly executed procedures. However, no technique can currently detect open-set procedural mistakes online. We propose PREGO, the first online one-class classification model for mistake detection in PRocedural EGOcentric videos. PREGO is based on an online action recognition component to model the current action, and a symbolic reasoning module to predict the next actions. Mistake detection is performed by comparing the recognized current action with the expected future one. We evaluate PREGO on two procedural egocentric video datasets, Assembly101 and Epic-tent, which we adapt for online benchmarking of procedural mistake detection to establish suitable benchmarks, thus defining the Assembly101-O and Epic-tent-O datasets, respectively. |
| title | PREGO: online mistake detection in PRocedural EGOcentric videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.01933 |