Challenge on Sound Scene Synthesis: Evaluating Text-to-Audio Generation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917813074001920 |
|---|---|
| author | Lee, Junwon Tailleur, Modan Heller, Laurie M. Choi, Keunwoo Lagrange, Mathieu McFee, Brian Imoto, Keisuke Okamoto, Yuki |
| author_facet | Lee, Junwon Tailleur, Modan Heller, Laurie M. Choi, Keunwoo Lagrange, Mathieu McFee, Brian Imoto, Keisuke Okamoto, Yuki |
| contents | Despite significant advancements in neural text-to-audio generation, challenges persist in controllability and evaluation. This paper addresses these issues through the Sound Scene Synthesis challenge held as part of the Detection and Classification of Acoustic Scenes and Events 2024. We present an evaluation protocol combining objective metric, namely Fréchet Audio Distance, with perceptual assessments, utilizing a structured prompt format to enable diverse captions and effective evaluation. Our analysis reveals varying performance across sound categories and model architectures, with larger models generally excelling but innovative lightweight approaches also showing promise. The strong correlation between objective metrics and human ratings validates our evaluation approach. We discuss outcomes in terms of audio quality, controllability, and architectural considerations for text-to-audio synthesizers, providing direction for future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_17589 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Challenge on Sound Scene Synthesis: Evaluating Text-to-Audio Generation Lee, Junwon Tailleur, Modan Heller, Laurie M. Choi, Keunwoo Lagrange, Mathieu McFee, Brian Imoto, Keisuke Okamoto, Yuki Sound Artificial Intelligence Machine Learning Multimedia Audio and Speech Processing Despite significant advancements in neural text-to-audio generation, challenges persist in controllability and evaluation. This paper addresses these issues through the Sound Scene Synthesis challenge held as part of the Detection and Classification of Acoustic Scenes and Events 2024. We present an evaluation protocol combining objective metric, namely Fréchet Audio Distance, with perceptual assessments, utilizing a structured prompt format to enable diverse captions and effective evaluation. Our analysis reveals varying performance across sound categories and model architectures, with larger models generally excelling but innovative lightweight approaches also showing promise. The strong correlation between objective metrics and human ratings validates our evaluation approach. We discuss outcomes in terms of audio quality, controllability, and architectural considerations for text-to-audio synthesizers, providing direction for future research. |
| title | Challenge on Sound Scene Synthesis: Evaluating Text-to-Audio Generation |
| topic | Sound Artificial Intelligence Machine Learning Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2410.17589 |