Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.10009 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910461487742976 |
|---|---|
| author | Manor, Hila Michaeli, Tomer |
| author_facet | Manor, Hila Michaeli, Tomer |
| contents | Editing signals using large pre-trained models, in a zero-shot manner, has recently seen rapid advancements in the image domain. However, this wave has yet to reach the audio domain. In this paper, we explore two zero-shot editing techniques for audio signals, which use DDPM inversion with pre-trained diffusion models. The first, which we coin ZEro-shot Text-based Audio (ZETA) editing, is adopted from the image domain. The second, named ZEro-shot UnSupervized (ZEUS) editing, is a novel approach for discovering semantically meaningful editing directions without supervision. When applied to music signals, this method exposes a range of musically interesting modifications, from controlling the participation of specific instruments to improvisations on the melody. Samples and code can be found in https://hilamanor.github.io/AudioEditing/ . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_10009 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion Manor, Hila Michaeli, Tomer Sound Machine Learning Audio and Speech Processing Editing signals using large pre-trained models, in a zero-shot manner, has recently seen rapid advancements in the image domain. However, this wave has yet to reach the audio domain. In this paper, we explore two zero-shot editing techniques for audio signals, which use DDPM inversion with pre-trained diffusion models. The first, which we coin ZEro-shot Text-based Audio (ZETA) editing, is adopted from the image domain. The second, named ZEro-shot UnSupervized (ZEUS) editing, is a novel approach for discovering semantically meaningful editing directions without supervision. When applied to music signals, this method exposes a range of musically interesting modifications, from controlling the participation of specific instruments to improvisations on the melody. Samples and code can be found in https://hilamanor.github.io/AudioEditing/ . |
| title | Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion |
| topic | Sound Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2402.10009 |