Exploring Meta Information for Audio-based Zero-shot Bird Classification
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909221386190848 |
|---|---|
| author | Gebhard, Alexander Triantafyllopoulos, Andreas Bez, Teresa Christ, Lukas Kathan, Alexander Schuller, Björn W. |
| author_facet | Gebhard, Alexander Triantafyllopoulos, Andreas Bez, Teresa Christ, Lukas Kathan, Alexander Schuller, Björn W. |
| contents | Advances in passive acoustic monitoring and machine learning have led to the procurement of vast datasets for computational bioacoustic research. Nevertheless, data scarcity is still an issue for rare and underrepresented species. This study investigates how meta-information can improve zero-shot audio classification, utilising bird species as an example case study due to the availability of rich and diverse meta-data. We investigate three different sources of metadata: textual bird sound descriptions encoded via (S)BERT, functional traits (AVONET), and bird life-history (BLH) characteristics. As audio features, we extract audio spectrogram transformer (AST) embeddings and project them to the dimension of the auxiliary information by adopting a single linear layer. Then, we employ the dot product as compatibility function and a standard zero-shot learning ranking hinge loss to determine the correct class. The best results are achieved by concatenating the AVONET and BLH features attaining a mean unweighted F1-score of .233 over five different test sets with 8 to 10 classes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_08398 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Exploring Meta Information for Audio-based Zero-shot Bird Classification Gebhard, Alexander Triantafyllopoulos, Andreas Bez, Teresa Christ, Lukas Kathan, Alexander Schuller, Björn W. Sound Machine Learning Audio and Speech Processing Advances in passive acoustic monitoring and machine learning have led to the procurement of vast datasets for computational bioacoustic research. Nevertheless, data scarcity is still an issue for rare and underrepresented species. This study investigates how meta-information can improve zero-shot audio classification, utilising bird species as an example case study due to the availability of rich and diverse meta-data. We investigate three different sources of metadata: textual bird sound descriptions encoded via (S)BERT, functional traits (AVONET), and bird life-history (BLH) characteristics. As audio features, we extract audio spectrogram transformer (AST) embeddings and project them to the dimension of the auxiliary information by adopting a single linear layer. Then, we employ the dot product as compatibility function and a standard zero-shot learning ranking hinge loss to determine the correct class. The best results are achieved by concatenating the AVONET and BLH features attaining a mean unweighted F1-score of .233 over five different test sets with 8 to 10 classes. |
| title | Exploring Meta Information for Audio-based Zero-shot Bird Classification |
| topic | Sound Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2309.08398 |