SAO-Instruct: Free-form Audio Editing using Natural Language Instructions
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911234106851328 |
|---|---|
| author | Ungersböck, Michael Grötschla, Florian Lanzendörfer, Luca A. Yi, June Young Choi, Changho Wattenhofer, Roger |
| author_facet | Ungersböck, Michael Grötschla, Florian Lanzendörfer, Luca A. Yi, June Young Choi, Changho Wattenhofer, Roger |
| contents | Generative models have made significant progress in synthesizing high-fidelity audio from short textual descriptions. However, editing existing audio using natural language has remained largely underexplored. Current approaches either require the complete description of the edited audio or are constrained to predefined edit instructions that lack flexibility. In this work, we introduce SAO-Instruct, a model based on Stable Audio Open capable of editing audio clips using any free-form natural language instruction. To train our model, we create a dataset of audio editing triplets (input audio, edit instruction, output audio) using Prompt-to-Prompt, DDPM inversion, and a manual editing pipeline. Although partially trained on synthetic data, our model generalizes well to real in-the-wild audio clips and unseen edit instructions. We demonstrate that SAO-Instruct achieves competitive performance on objective metrics and outperforms other audio editing approaches in a subjective listening study. To encourage future research, we release our code and model weights. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_22795 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SAO-Instruct: Free-form Audio Editing using Natural Language Instructions Ungersböck, Michael Grötschla, Florian Lanzendörfer, Luca A. Yi, June Young Choi, Changho Wattenhofer, Roger Sound Machine Learning Generative models have made significant progress in synthesizing high-fidelity audio from short textual descriptions. However, editing existing audio using natural language has remained largely underexplored. Current approaches either require the complete description of the edited audio or are constrained to predefined edit instructions that lack flexibility. In this work, we introduce SAO-Instruct, a model based on Stable Audio Open capable of editing audio clips using any free-form natural language instruction. To train our model, we create a dataset of audio editing triplets (input audio, edit instruction, output audio) using Prompt-to-Prompt, DDPM inversion, and a manual editing pipeline. Although partially trained on synthetic data, our model generalizes well to real in-the-wild audio clips and unseen edit instructions. We demonstrate that SAO-Instruct achieves competitive performance on objective metrics and outperforms other audio editing approaches in a subjective listening study. To encourage future research, we release our code and model weights. |
| title | SAO-Instruct: Free-form Audio Editing using Natural Language Instructions |
| topic | Sound Machine Learning |
| url | https://arxiv.org/abs/2510.22795 |