Leveraging Pre-Trained Autoencoders for Interpretable Prototype Learning of Music Audio
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914679190716416 |
|---|---|
| author | Alonso-Jiménez, Pablo Pepino, Leonardo Batlle-Roca, Roser Zinemanas, Pablo Bogdanov, Dmitry Serra, Xavier Rocamora, Martín |
| author_facet | Alonso-Jiménez, Pablo Pepino, Leonardo Batlle-Roca, Roser Zinemanas, Pablo Bogdanov, Dmitry Serra, Xavier Rocamora, Martín |
| contents | We present PECMAE, an interpretable model for music audio classification based on prototype learning. Our model is based on a previous method, APNet, which jointly learns an autoencoder and a prototypical network. Instead, we propose to decouple both training processes. This enables us to leverage existing self-supervised autoencoders pre-trained on much larger data (EnCodecMAE), providing representations with better generalization. APNet allows prototypes' reconstruction to waveforms for interpretability relying on the nearest training data samples. In contrast, we explore using a diffusion decoder that allows reconstruction without such dependency. We evaluate our method on datasets for music instrument classification (Medley-Solos-DB) and genre recognition (GTZAN and a larger in-house dataset), the latter being a more challenging task not addressed with prototypical networks before. We find that the prototype-based models preserve most of the performance achieved with the autoencoder embeddings, while the sonification of prototypes benefits understanding the behavior of the classifier. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_09318 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Leveraging Pre-Trained Autoencoders for Interpretable Prototype Learning of Music Audio Alonso-Jiménez, Pablo Pepino, Leonardo Batlle-Roca, Roser Zinemanas, Pablo Bogdanov, Dmitry Serra, Xavier Rocamora, Martín Sound Artificial Intelligence Multimedia Audio and Speech Processing We present PECMAE, an interpretable model for music audio classification based on prototype learning. Our model is based on a previous method, APNet, which jointly learns an autoencoder and a prototypical network. Instead, we propose to decouple both training processes. This enables us to leverage existing self-supervised autoencoders pre-trained on much larger data (EnCodecMAE), providing representations with better generalization. APNet allows prototypes' reconstruction to waveforms for interpretability relying on the nearest training data samples. In contrast, we explore using a diffusion decoder that allows reconstruction without such dependency. We evaluate our method on datasets for music instrument classification (Medley-Solos-DB) and genre recognition (GTZAN and a larger in-house dataset), the latter being a more challenging task not addressed with prototypical networks before. We find that the prototype-based models preserve most of the performance achieved with the autoencoder embeddings, while the sonification of prototypes benefits understanding the behavior of the classifier. |
| title | Leveraging Pre-Trained Autoencoders for Interpretable Prototype Learning of Music Audio |
| topic | Sound Artificial Intelligence Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2402.09318 |