Diff-TONE: Timestep Optimization for iNstrument Editing in Text-to-Music Diffusion Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912438336618496 |
|---|---|
| author | Baoueb, Teysir Bie, Xiaoyu Wang, Xi Richard, Gaël |
| author_facet | Baoueb, Teysir Bie, Xiaoyu Wang, Xi Richard, Gaël |
| contents | Breakthroughs in text-to-music generation models are transforming the creative landscape, equipping musicians with innovative tools for composition and experimentation like never before. However, controlling the generation process to achieve a specific desired outcome remains a significant challenge. Even a minor change in the text prompt, combined with the same random seed, can drastically alter the generated piece. In this paper, we explore the application of existing text-to-music diffusion models for instrument editing. Specifically, for an existing audio track, we aim to leverage a pretrained text-to-music diffusion model to edit the instrument while preserving the underlying content. Based on the insight that the model first focuses on the overall structure or content of the audio, then adds instrument information, and finally refines the quality, we show that selecting a well-chosen intermediate timestep, identified through an instrument classifier, yields a balance between preserving the original piece's content and achieving the desired timbre. Our method does not require additional training of the text-to-music diffusion model, nor does it compromise the generation process's speed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_15530 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Diff-TONE: Timestep Optimization for iNstrument Editing in Text-to-Music Diffusion Models Baoueb, Teysir Bie, Xiaoyu Wang, Xi Richard, Gaël Sound Machine Learning Audio and Speech Processing Signal Processing Breakthroughs in text-to-music generation models are transforming the creative landscape, equipping musicians with innovative tools for composition and experimentation like never before. However, controlling the generation process to achieve a specific desired outcome remains a significant challenge. Even a minor change in the text prompt, combined with the same random seed, can drastically alter the generated piece. In this paper, we explore the application of existing text-to-music diffusion models for instrument editing. Specifically, for an existing audio track, we aim to leverage a pretrained text-to-music diffusion model to edit the instrument while preserving the underlying content. Based on the insight that the model first focuses on the overall structure or content of the audio, then adds instrument information, and finally refines the quality, we show that selecting a well-chosen intermediate timestep, identified through an instrument classifier, yields a balance between preserving the original piece's content and achieving the desired timbre. Our method does not require additional training of the text-to-music diffusion model, nor does it compromise the generation process's speed. |
| title | Diff-TONE: Timestep Optimization for iNstrument Editing in Text-to-Music Diffusion Models |
| topic | Sound Machine Learning Audio and Speech Processing Signal Processing |
| url | https://arxiv.org/abs/2506.15530 |