Detecting Music Performance Errors with Transformers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916550705938432 |
|---|---|
| author | Chou, Benjamin Shiue-Hal Jajal, Purvish Eliopoulos, Nicholas John Nadolsky, Tim Yang, Cheng-Yun Ravi, Nikita Davis, James C. Yun, Kristen Yeon-Ji Lu, Yung-Hsiang |
| author_facet | Chou, Benjamin Shiue-Hal Jajal, Purvish Eliopoulos, Nicholas John Nadolsky, Tim Yang, Cheng-Yun Ravi, Nikita Davis, James C. Yun, Kristen Yeon-Ji Lu, Yung-Hsiang |
| contents | Beginner musicians often struggle to identify specific errors in their performances, such as playing incorrect notes or rhythms. There are two limitations in existing tools for music error detection: (1) Existing approaches rely on automatic alignment; therefore, they are prone to errors caused by small deviations between alignment targets.; (2) There is a lack of sufficient data to train music error detection models, resulting in over-reliance on heuristics. To address (1), we propose a novel transformer model, Polytune, that takes audio inputs and outputs annotated music scores. This model can be trained end-to-end to implicitly align and compare performance audio with music scores through latent space representations. To address (2), we present a novel data generation technique capable of creating large-scale synthetic music error datasets. Our approach achieves a 64.1% average Error Detection F1 score, improving upon prior work by 40 percentage points across 14 instruments. Additionally, compared with existing transcription methods repurposed for music error detection, our model can handle multiple instruments. Our source code and datasets are available at https://github.com/ben2002chou/Polytune. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_02030 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Detecting Music Performance Errors with Transformers Chou, Benjamin Shiue-Hal Jajal, Purvish Eliopoulos, Nicholas John Nadolsky, Tim Yang, Cheng-Yun Ravi, Nikita Davis, James C. Yun, Kristen Yeon-Ji Lu, Yung-Hsiang Sound Artificial Intelligence Audio and Speech Processing Beginner musicians often struggle to identify specific errors in their performances, such as playing incorrect notes or rhythms. There are two limitations in existing tools for music error detection: (1) Existing approaches rely on automatic alignment; therefore, they are prone to errors caused by small deviations between alignment targets.; (2) There is a lack of sufficient data to train music error detection models, resulting in over-reliance on heuristics. To address (1), we propose a novel transformer model, Polytune, that takes audio inputs and outputs annotated music scores. This model can be trained end-to-end to implicitly align and compare performance audio with music scores through latent space representations. To address (2), we present a novel data generation technique capable of creating large-scale synthetic music error datasets. Our approach achieves a 64.1% average Error Detection F1 score, improving upon prior work by 40 percentage points across 14 instruments. Additionally, compared with existing transcription methods repurposed for music error detection, our model can handle multiple instruments. Our source code and datasets are available at https://github.com/ben2002chou/Polytune. |
| title | Detecting Music Performance Errors with Transformers |
| topic | Sound Artificial Intelligence Audio and Speech Processing |
| url | https://arxiv.org/abs/2501.02030 |