Token-Based Audio Inpainting via Discrete Diffusion
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910024820850688 |
|---|---|
| author | Dror, Tali Shoham, Iftach Buchris, Moshe Gal, Oren Permuter, Haim Katz, Gilad Nachmani, Eliya |
| author_facet | Dror, Tali Shoham, Iftach Buchris, Moshe Gal, Oren Permuter, Haim Katz, Gilad Nachmani, Eliya |
| contents | Audio inpainting seeks to restore missing segments in degraded recordings. Previous diffusion-based methods exhibit impaired performance when the missing region is large. We introduce the first approach that applies discrete diffusion over tokenized music representations from a pre-trained audio tokenizer, enabling stable and semantically coherent restoration of long gaps. Our method further incorporates two training approaches: a derivative-based regularization loss that enforces smooth temporal dynamics, and a span-based absorbing transition that provides structured corruption during diffusion. Experiments on the MusicNet and MAESTRO datasets with gaps up to 750 ms show that our approach consistently outperforms strong baselines across range of gap lengths, for gaps of 150 ms and above. This work advances musical audio restoration and introduces new directions for discrete diffusion model training. Visit our project page for examples and code. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_08333 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Token-Based Audio Inpainting via Discrete Diffusion Dror, Tali Shoham, Iftach Buchris, Moshe Gal, Oren Permuter, Haim Katz, Gilad Nachmani, Eliya Sound Artificial Intelligence Information Theory Machine Learning Audio and Speech Processing Audio inpainting seeks to restore missing segments in degraded recordings. Previous diffusion-based methods exhibit impaired performance when the missing region is large. We introduce the first approach that applies discrete diffusion over tokenized music representations from a pre-trained audio tokenizer, enabling stable and semantically coherent restoration of long gaps. Our method further incorporates two training approaches: a derivative-based regularization loss that enforces smooth temporal dynamics, and a span-based absorbing transition that provides structured corruption during diffusion. Experiments on the MusicNet and MAESTRO datasets with gaps up to 750 ms show that our approach consistently outperforms strong baselines across range of gap lengths, for gaps of 150 ms and above. This work advances musical audio restoration and introduces new directions for discrete diffusion model training. Visit our project page for examples and code. |
| title | Token-Based Audio Inpainting via Discrete Diffusion |
| topic | Sound Artificial Intelligence Information Theory Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2507.08333 |