LVNS-RAVE: Diversified audio generation with RAVE and Latent Vector Novelty Search
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909177487556608 |
|---|---|
| author | Guo, Jinyue Christodoulou, Anna-Maria Laczko, Balint Glette, Kyrre |
| author_facet | Guo, Jinyue Christodoulou, Anna-Maria Laczko, Balint Glette, Kyrre |
| contents | Evolutionary Algorithms and Generative Deep Learning have been two of the most powerful tools for sound generation tasks. However, they have limitations: Evolutionary Algorithms require complicated designs, posing challenges in control and achieving realistic sound generation. Generative Deep Learning models often copy from the dataset and lack creativity. In this paper, we propose LVNS-RAVE, a method to combine Evolutionary Algorithms and Generative Deep Learning to produce realistic and novel sounds. We use the RAVE model as the sound generator and the VGGish model as a novelty evaluator in the Latent Vector Novelty Search (LVNS) algorithm. The reported experiments show that the method can successfully generate diversified, novel audio samples under different mutation setups using different pre-trained RAVE models. The characteristics of the generation process can be easily controlled with the mutation parameters. The proposed algorithm can be a creative tool for sound artists and musicians. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_14063 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LVNS-RAVE: Diversified audio generation with RAVE and Latent Vector Novelty Search Guo, Jinyue Christodoulou, Anna-Maria Laczko, Balint Glette, Kyrre Sound Machine Learning Neural and Evolutionary Computing Audio and Speech Processing Evolutionary Algorithms and Generative Deep Learning have been two of the most powerful tools for sound generation tasks. However, they have limitations: Evolutionary Algorithms require complicated designs, posing challenges in control and achieving realistic sound generation. Generative Deep Learning models often copy from the dataset and lack creativity. In this paper, we propose LVNS-RAVE, a method to combine Evolutionary Algorithms and Generative Deep Learning to produce realistic and novel sounds. We use the RAVE model as the sound generator and the VGGish model as a novelty evaluator in the Latent Vector Novelty Search (LVNS) algorithm. The reported experiments show that the method can successfully generate diversified, novel audio samples under different mutation setups using different pre-trained RAVE models. The characteristics of the generation process can be easily controlled with the mutation parameters. The proposed algorithm can be a creative tool for sound artists and musicians. |
| title | LVNS-RAVE: Diversified audio generation with RAVE and Latent Vector Novelty Search |
| topic | Sound Machine Learning Neural and Evolutionary Computing Audio and Speech Processing |
| url | https://arxiv.org/abs/2404.14063 |