Timbre-Trap: A Low-Resource Framework for Instrument-Agnostic Music Transcription
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929221694128128 |
|---|---|
| author | Cwitkowitz, Frank Cheuk, Kin Wai Choi, Woosung Martínez-Ramírez, Marco A. Toyama, Keisuke Liao, Wei-Hsiang Mitsufuji, Yuki |
| author_facet | Cwitkowitz, Frank Cheuk, Kin Wai Choi, Woosung Martínez-Ramírez, Marco A. Toyama, Keisuke Liao, Wei-Hsiang Mitsufuji, Yuki |
| contents | In recent years, research on music transcription has focused mainly on architecture design and instrument-specific data acquisition. With the lack of availability of diverse datasets, progress is often limited to solo-instrument tasks such as piano transcription. Several works have explored multi-instrument transcription as a means to bolster the performance of models on low-resource tasks, but these methods face the same data availability issues. We propose Timbre-Trap, a novel framework which unifies music transcription and audio reconstruction by exploiting the strong separability between pitch and timbre. We train a single autoencoder to simultaneously estimate pitch salience and reconstruct complex spectral coefficients, selecting between either output during the decoding stage via a simple switch mechanism. In this way, the model learns to produce coefficients corresponding to timbre-less audio, which can be interpreted as pitch salience. We demonstrate that the framework leads to performance comparable to state-of-the-art instrument-agnostic transcription methods, while only requiring a small amount of annotated data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_15717 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Timbre-Trap: A Low-Resource Framework for Instrument-Agnostic Music Transcription Cwitkowitz, Frank Cheuk, Kin Wai Choi, Woosung Martínez-Ramírez, Marco A. Toyama, Keisuke Liao, Wei-Hsiang Mitsufuji, Yuki Audio and Speech Processing Machine Learning Sound In recent years, research on music transcription has focused mainly on architecture design and instrument-specific data acquisition. With the lack of availability of diverse datasets, progress is often limited to solo-instrument tasks such as piano transcription. Several works have explored multi-instrument transcription as a means to bolster the performance of models on low-resource tasks, but these methods face the same data availability issues. We propose Timbre-Trap, a novel framework which unifies music transcription and audio reconstruction by exploiting the strong separability between pitch and timbre. We train a single autoencoder to simultaneously estimate pitch salience and reconstruct complex spectral coefficients, selecting between either output during the decoding stage via a simple switch mechanism. In this way, the model learns to produce coefficients corresponding to timbre-less audio, which can be interpreted as pitch salience. We demonstrate that the framework leads to performance comparable to state-of-the-art instrument-agnostic transcription methods, while only requiring a small amount of annotated data. |
| title | Timbre-Trap: A Low-Resource Framework for Instrument-Agnostic Music Transcription |
| topic | Audio and Speech Processing Machine Learning Sound |
| url | https://arxiv.org/abs/2309.15717 |