MR-MT3: Memory Retaining Multi-Track Music Transcription to Mitigate Instrument Leakage
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910368290308096 |
|---|---|
| author | Tan, Hao Hao Cheuk, Kin Wai Cho, Taemin Liao, Wei-Hsiang Mitsufuji, Yuki |
| author_facet | Tan, Hao Hao Cheuk, Kin Wai Cho, Taemin Liao, Wei-Hsiang Mitsufuji, Yuki |
| contents | This paper presents enhancements to the MT3 model, a state-of-the-art (SOTA) token-based multi-instrument automatic music transcription (AMT) model. Despite SOTA performance, MT3 has the issue of instrument leakage, where transcriptions are fragmented across different instruments. To mitigate this, we propose MR-MT3, with enhancements including a memory retention mechanism, prior token sampling, and token shuffling are proposed. These methods are evaluated on the Slakh2100 dataset, demonstrating improved onset F1 scores and reduced instrument leakage. In addition to the conventional multi-instrument transcription F1 score, new metrics such as the instrument leakage ratio and the instrument detection F1 score are introduced for a more comprehensive assessment of transcription quality. The study also explores the issue of domain overfitting by evaluating MT3 on single-instrument monophonic datasets such as ComMU and NSynth. The findings, along with the source code, are shared to facilitate future work aimed at refining token-based multi-instrument AMT models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_10024 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MR-MT3: Memory Retaining Multi-Track Music Transcription to Mitigate Instrument Leakage Tan, Hao Hao Cheuk, Kin Wai Cho, Taemin Liao, Wei-Hsiang Mitsufuji, Yuki Sound Artificial Intelligence Machine Learning Multimedia Audio and Speech Processing This paper presents enhancements to the MT3 model, a state-of-the-art (SOTA) token-based multi-instrument automatic music transcription (AMT) model. Despite SOTA performance, MT3 has the issue of instrument leakage, where transcriptions are fragmented across different instruments. To mitigate this, we propose MR-MT3, with enhancements including a memory retention mechanism, prior token sampling, and token shuffling are proposed. These methods are evaluated on the Slakh2100 dataset, demonstrating improved onset F1 scores and reduced instrument leakage. In addition to the conventional multi-instrument transcription F1 score, new metrics such as the instrument leakage ratio and the instrument detection F1 score are introduced for a more comprehensive assessment of transcription quality. The study also explores the issue of domain overfitting by evaluating MT3 on single-instrument monophonic datasets such as ComMU and NSynth. The findings, along with the source code, are shared to facilitate future work aimed at refining token-based multi-instrument AMT models. |
| title | MR-MT3: Memory Retaining Multi-Track Music Transcription to Mitigate Instrument Leakage |
| topic | Sound Artificial Intelligence Machine Learning Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2403.10024 |