Break-the-Beat! Controllable MIDI-to-Drum Audio Synthesis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917495472914432 |
|---|---|
| author | Cui, Shuyang Zhong, Zhi Wu, Qiyu Novack, Zachary Choi, Woosung Toyama, Keisuke Cheuk, Kin Wai Koo, Junghyun Ikemiya, Yukara Simon, Christian Nagashima, Chihiro Takahashi, Shusuke |
| author_facet | Cui, Shuyang Zhong, Zhi Wu, Qiyu Novack, Zachary Choi, Woosung Toyama, Keisuke Cheuk, Kin Wai Koo, Junghyun Ikemiya, Yukara Simon, Christian Nagashima, Chihiro Takahashi, Shusuke |
| contents | Current methods for creating drum loop audio in digital music production, such as using one-shot samples or resampling, often demand non-trivial efforts of creators. While recent generative models achieve high fidelity and adhere to text, they lack the specific control needed for such a task. Existing symbolic-to-audio research often focuses on single, tonal instruments, leaving the challenge of polyphonic, percussive drum synthesis unaddressed. We address this gap by introducing ``Break-the-Beat!,'' a model capable of rendering a drum MIDI with the timbre of a reference audio. It is built by fine-tuning a pre-trained text-to-audio model with our proposed content encoder and a effective hybrid conditioning mechanism. To enable this, we construct a new dataset of paired target-reference drum audio from existing drum audio datasets. Experiments demonstrate that our model generates high-quality drum audio that follows high-resolution drum MIDI, achieving strong performance across metrics of audio quality, rhythmic alignment, and beat continuity. This offer producers a new, controllable tool for creative production. Demo page: https://ik4sumii.github.io/break-the-beat/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_14555 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Break-the-Beat! Controllable MIDI-to-Drum Audio Synthesis Cui, Shuyang Zhong, Zhi Wu, Qiyu Novack, Zachary Choi, Woosung Toyama, Keisuke Cheuk, Kin Wai Koo, Junghyun Ikemiya, Yukara Simon, Christian Nagashima, Chihiro Takahashi, Shusuke Sound Artificial Intelligence Current methods for creating drum loop audio in digital music production, such as using one-shot samples or resampling, often demand non-trivial efforts of creators. While recent generative models achieve high fidelity and adhere to text, they lack the specific control needed for such a task. Existing symbolic-to-audio research often focuses on single, tonal instruments, leaving the challenge of polyphonic, percussive drum synthesis unaddressed. We address this gap by introducing ``Break-the-Beat!,'' a model capable of rendering a drum MIDI with the timbre of a reference audio. It is built by fine-tuning a pre-trained text-to-audio model with our proposed content encoder and a effective hybrid conditioning mechanism. To enable this, we construct a new dataset of paired target-reference drum audio from existing drum audio datasets. Experiments demonstrate that our model generates high-quality drum audio that follows high-resolution drum MIDI, achieving strong performance across metrics of audio quality, rhythmic alignment, and beat continuity. This offer producers a new, controllable tool for creative production. Demo page: https://ik4sumii.github.io/break-the-beat/ |
| title | Break-the-Beat! Controllable MIDI-to-Drum Audio Synthesis |
| topic | Sound Artificial Intelligence |
| url | https://arxiv.org/abs/2605.14555 |