SONIQUE: Video Background Music Generation Using Unpaired Audio-Visual Data
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916630132424704 |
|---|---|
| author | Zhang, Liqian Fuentes, Magdalena |
| author_facet | Zhang, Liqian Fuentes, Magdalena |
| contents | We present SONIQUE, a model for generating background music tailored to video content. Unlike traditional video-to-music generation approaches, which rely heavily on paired audio-visual datasets, SONIQUE leverages unpaired data, combining royalty-free music and independent video sources. By utilizing large language models (LLMs) for video understanding and converting visual descriptions into musical tags, alongside a U-Net-based conditional diffusion model, SONIQUE enables customizable music generation. Users can control specific aspects of the music, such as instruments, genres, tempo, and melodies, ensuring the generated output fits their creative vision. SONIQUE is open-source, with a demo available online. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03879 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SONIQUE: Video Background Music Generation Using Unpaired Audio-Visual Data Zhang, Liqian Fuentes, Magdalena Sound Multimedia Audio and Speech Processing We present SONIQUE, a model for generating background music tailored to video content. Unlike traditional video-to-music generation approaches, which rely heavily on paired audio-visual datasets, SONIQUE leverages unpaired data, combining royalty-free music and independent video sources. By utilizing large language models (LLMs) for video understanding and converting visual descriptions into musical tags, alongside a U-Net-based conditional diffusion model, SONIQUE enables customizable music generation. Users can control specific aspects of the music, such as instruments, genres, tempo, and melodies, ensuring the generated output fits their creative vision. SONIQUE is open-source, with a demo available online. |
| title | SONIQUE: Video Background Music Generation Using Unpaired Audio-Visual Data |
| topic | Sound Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2410.03879 |