Bird Vocalization Embedding Extraction Using Self-Supervised Disentangled Representation Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916545106542592 |
|---|---|
| author | Shi, Runwu Itoyama, Katsutoshi Nakadai, Kazuhiro |
| author_facet | Shi, Runwu Itoyama, Katsutoshi Nakadai, Kazuhiro |
| contents | This paper addresses the extraction of the bird vocalization embedding from the whole song level using disentangled representation learning (DRL). Bird vocalization embeddings are necessary for large-scale bioacoustic tasks, and self-supervised methods such as Variational Autoencoder (VAE) have shown their performance in extracting such low-dimensional embeddings from vocalization segments on the note or syllable level. To extend the processing level to the entire song instead of cutting into segments, this paper regards each vocalization as the generalized and discriminative part and uses two encoders to learn these two parts. The proposed method is evaluated on the Great Tits dataset according to the clustering performance, and the results outperform the compared pre-trained models and vanilla VAE. Finally, this paper analyzes the informative part of the embedding, further compresses its dimension, and explains the disentangled performance of bird vocalizations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_20146 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Bird Vocalization Embedding Extraction Using Self-Supervised Disentangled Representation Learning Shi, Runwu Itoyama, Katsutoshi Nakadai, Kazuhiro Audio and Speech Processing Sound Signal Processing This paper addresses the extraction of the bird vocalization embedding from the whole song level using disentangled representation learning (DRL). Bird vocalization embeddings are necessary for large-scale bioacoustic tasks, and self-supervised methods such as Variational Autoencoder (VAE) have shown their performance in extracting such low-dimensional embeddings from vocalization segments on the note or syllable level. To extend the processing level to the entire song instead of cutting into segments, this paper regards each vocalization as the generalized and discriminative part and uses two encoders to learn these two parts. The proposed method is evaluated on the Great Tits dataset according to the clustering performance, and the results outperform the compared pre-trained models and vanilla VAE. Finally, this paper analyzes the informative part of the embedding, further compresses its dimension, and explains the disentangled performance of bird vocalizations. |
| title | Bird Vocalization Embedding Extraction Using Self-Supervised Disentangled Representation Learning |
| topic | Audio and Speech Processing Sound Signal Processing |
| url | https://arxiv.org/abs/2412.20146 |