How to Track and Segment Fish without Human Annotations: A Self-Supervised Deep Learning Approach
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910844321792000 |
|---|---|
| author | Saleh, Alzayat Sheaves, Marcus Jerry, Dean Azghadi, Mostafa Rahimi |
| author_facet | Saleh, Alzayat Sheaves, Marcus Jerry, Dean Azghadi, Mostafa Rahimi |
| contents | Tracking fish movements and sizes of fish is crucial to understanding their ecology and behaviour. Knowing where fish migrate, how they interact with their environment, and how their size affects their behaviour can help ecologists develop more effective conservation and management strategies to protect fish populations and their habitats. Deep learning is a promising tool to analyze fish ecology from underwater videos. However, training deep neural networks (DNNs) for fish tracking and segmentation requires high-quality labels, which are expensive to obtain. We propose an alternative unsupervised approach that relies on spatial and temporal variations in video data to generate noisy pseudo-ground-truth labels. We train a multitask DNN using these pseudo-labels. Our framework consists of three stages: (1) an optical flow model generates the pseudo labels using spatial and temporal consistency between frames, (2) a self-supervised model refines the pseudo-labels incrementally, and (3) a segmentation network uses the refined labels for training. Consequently, we perform extensive experiments to validate our method on three public underwater video datasets and demonstrate its effectiveness for video annotation and segmentation. We also evaluate its robustness to different imaging conditions and discuss its limitations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2208_10662 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | How to Track and Segment Fish without Human Annotations: A Self-Supervised Deep Learning Approach Saleh, Alzayat Sheaves, Marcus Jerry, Dean Azghadi, Mostafa Rahimi Computer Vision and Pattern Recognition Tracking fish movements and sizes of fish is crucial to understanding their ecology and behaviour. Knowing where fish migrate, how they interact with their environment, and how their size affects their behaviour can help ecologists develop more effective conservation and management strategies to protect fish populations and their habitats. Deep learning is a promising tool to analyze fish ecology from underwater videos. However, training deep neural networks (DNNs) for fish tracking and segmentation requires high-quality labels, which are expensive to obtain. We propose an alternative unsupervised approach that relies on spatial and temporal variations in video data to generate noisy pseudo-ground-truth labels. We train a multitask DNN using these pseudo-labels. Our framework consists of three stages: (1) an optical flow model generates the pseudo labels using spatial and temporal consistency between frames, (2) a self-supervised model refines the pseudo-labels incrementally, and (3) a segmentation network uses the refined labels for training. Consequently, we perform extensive experiments to validate our method on three public underwater video datasets and demonstrate its effectiveness for video annotation and segmentation. We also evaluate its robustness to different imaging conditions and discuss its limitations. |
| title | How to Track and Segment Fish without Human Annotations: A Self-Supervised Deep Learning Approach |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2208.10662 |