A study of animal action segmentation algorithms across supervised, unsupervised, and semi-supervised learning paradigms
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916529352736768 |
|---|---|
| author | Blau, Ari Schaffer, Evan S Mishra, Neeli Miska, Nathaniel J Laboratory, The International Brain Paninski, Liam Whiteway, Matthew R |
| author_facet | Blau, Ari Schaffer, Evan S Mishra, Neeli Miska, Nathaniel J Laboratory, The International Brain Paninski, Liam Whiteway, Matthew R |
| contents | Action segmentation of behavioral videos is the process of labeling each frame as belonging to one or more discrete classes, and is a crucial component of many studies that investigate animal behavior. A wide range of algorithms exist to automatically parse discrete animal behavior, encompassing supervised, unsupervised, and semi-supervised learning paradigms. These algorithms -- which include tree-based models, deep neural networks, and graphical models -- differ widely in their structure and assumptions on the data. Using four datasets spanning multiple species -- fly, mouse, and human -- we systematically study how the outputs of these various algorithms align with manually annotated behaviors of interest. Along the way, we introduce a semi-supervised action segmentation model that bridges the gap between supervised deep neural networks and unsupervised graphical models. We find that fully supervised temporal convolutional networks with the addition of temporal information in the observations perform the best on our supervised metrics across all datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_16727 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A study of animal action segmentation algorithms across supervised, unsupervised, and semi-supervised learning paradigms Blau, Ari Schaffer, Evan S Mishra, Neeli Miska, Nathaniel J Laboratory, The International Brain Paninski, Liam Whiteway, Matthew R Computer Vision and Pattern Recognition Quantitative Methods Action segmentation of behavioral videos is the process of labeling each frame as belonging to one or more discrete classes, and is a crucial component of many studies that investigate animal behavior. A wide range of algorithms exist to automatically parse discrete animal behavior, encompassing supervised, unsupervised, and semi-supervised learning paradigms. These algorithms -- which include tree-based models, deep neural networks, and graphical models -- differ widely in their structure and assumptions on the data. Using four datasets spanning multiple species -- fly, mouse, and human -- we systematically study how the outputs of these various algorithms align with manually annotated behaviors of interest. Along the way, we introduce a semi-supervised action segmentation model that bridges the gap between supervised deep neural networks and unsupervised graphical models. We find that fully supervised temporal convolutional networks with the addition of temporal information in the observations perform the best on our supervised metrics across all datasets. |
| title | A study of animal action segmentation algorithms across supervised, unsupervised, and semi-supervised learning paradigms |
| topic | Computer Vision and Pattern Recognition Quantitative Methods |
| url | https://arxiv.org/abs/2407.16727 |