Enhancing Audio Generation Diversity with Visual Information
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910350781186048 |
|---|---|
| author | Xie, Zeyu Li, Baihan Xu, Xuenan Wu, Mengyue Yu, Kai |
| author_facet | Xie, Zeyu Li, Baihan Xu, Xuenan Wu, Mengyue Yu, Kai |
| contents | Audio and sound generation has garnered significant attention in recent years, with a primary focus on improving the quality of generated audios. However, there has been limited research on enhancing the diversity of generated audio, particularly when it comes to audio generation within specific categories. Current models tend to produce homogeneous audio samples within a category. This work aims to address this limitation by improving the diversity of generated audio with visual information. We propose a clustering-based method, leveraging visual information to guide the model in generating distinct audio content within each category. Results on seven categories indicate that extra visual input can largely enhance audio generation diversity. Audio samples are available at https://zeyuxie29.github.io/DiverseAudioGeneration. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_01278 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Enhancing Audio Generation Diversity with Visual Information Xie, Zeyu Li, Baihan Xu, Xuenan Wu, Mengyue Yu, Kai Sound Audio and Speech Processing I.2 Audio and sound generation has garnered significant attention in recent years, with a primary focus on improving the quality of generated audios. However, there has been limited research on enhancing the diversity of generated audio, particularly when it comes to audio generation within specific categories. Current models tend to produce homogeneous audio samples within a category. This work aims to address this limitation by improving the diversity of generated audio with visual information. We propose a clustering-based method, leveraging visual information to guide the model in generating distinct audio content within each category. Results on seven categories indicate that extra visual input can largely enhance audio generation diversity. Audio samples are available at https://zeyuxie29.github.io/DiverseAudioGeneration. |
| title | Enhancing Audio Generation Diversity with Visual Information |
| topic | Sound Audio and Speech Processing I.2 |
| url | https://arxiv.org/abs/2403.01278 |