SynthScribe: Deep Multimodal Tools for Synthesizer Sound Retrieval and Exploration
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910338483486720 |
|---|---|
| author | Brade, Stephen Wang, Bryan Sousa, Mauricio Newsome, Gregory Lee Oore, Sageev Grossman, Tovi |
| author_facet | Brade, Stephen Wang, Bryan Sousa, Mauricio Newsome, Gregory Lee Oore, Sageev Grossman, Tovi |
| contents | Synthesizers are powerful tools that allow musicians to create dynamic and original sounds. Existing commercial interfaces for synthesizers typically require musicians to interact with complex low-level parameters or to manage large libraries of premade sounds. To address these challenges, we implement SynthScribe -- a fullstack system that uses multimodal deep learning to let users express their intentions at a much higher level. We implement features which address a number of difficulties, namely 1) searching through existing sounds, 2) creating completely new sounds, 3) making meaningful modifications to a given sound. This is achieved with three main features: a multimodal search engine for a large library of synthesizer sounds; a user centered genetic algorithm by which completely new sounds can be created and selected given the users preferences; a sound editing support feature which highlights and gives examples for key control parameters with respect to a text or audio based query. The results of our user studies show SynthScribe is capable of reliably retrieving and modifying sounds while also affording the ability to create completely new sounds that expand a musicians creative horizon. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_04690 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | SynthScribe: Deep Multimodal Tools for Synthesizer Sound Retrieval and Exploration Brade, Stephen Wang, Bryan Sousa, Mauricio Newsome, Gregory Lee Oore, Sageev Grossman, Tovi Human-Computer Interaction Artificial Intelligence Sound Audio and Speech Processing Synthesizers are powerful tools that allow musicians to create dynamic and original sounds. Existing commercial interfaces for synthesizers typically require musicians to interact with complex low-level parameters or to manage large libraries of premade sounds. To address these challenges, we implement SynthScribe -- a fullstack system that uses multimodal deep learning to let users express their intentions at a much higher level. We implement features which address a number of difficulties, namely 1) searching through existing sounds, 2) creating completely new sounds, 3) making meaningful modifications to a given sound. This is achieved with three main features: a multimodal search engine for a large library of synthesizer sounds; a user centered genetic algorithm by which completely new sounds can be created and selected given the users preferences; a sound editing support feature which highlights and gives examples for key control parameters with respect to a text or audio based query. The results of our user studies show SynthScribe is capable of reliably retrieving and modifying sounds while also affording the ability to create completely new sounds that expand a musicians creative horizon. |
| title | SynthScribe: Deep Multimodal Tools for Synthesizer Sound Retrieval and Exploration |
| topic | Human-Computer Interaction Artificial Intelligence Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2312.04690 |