Jointly Recognizing Speech and Singing Voices Based on Multi-Task Audio Source Separation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911843963895808 |
|---|---|
| author | Bai, Ye Li, Chenxing Li, Hao Zhao, Yuanyuan Wang, Xiaorui |
| author_facet | Bai, Ye Li, Chenxing Li, Hao Zhao, Yuanyuan Wang, Xiaorui |
| contents | In short video and live broadcasts, speech, singing voice, and background music often overlap and obscure each other. This complexity creates difficulties in structuring and recognizing the audio content, which may impair subsequent ASR and music understanding applications. This paper proposes a multi-task audio source separation (MTASS) based ASR model called JRSV, which Jointly Recognizes Speech and singing Voices. Specifically, the MTASS module separates the mixed audio into distinct speech and singing voice tracks while removing background music. The CTC/attention hybrid recognition module recognizes both tracks. Online distillation is proposed to improve the robustness of recognition further. To evaluate the proposed methods, a benchmark dataset is constructed and released. Experimental results demonstrate that JRSV can significantly improve recognition accuracy on each track of the mixed audio. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_11275 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Jointly Recognizing Speech and Singing Voices Based on Multi-Task Audio Source Separation Bai, Ye Li, Chenxing Li, Hao Zhao, Yuanyuan Wang, Xiaorui Sound Audio and Speech Processing In short video and live broadcasts, speech, singing voice, and background music often overlap and obscure each other. This complexity creates difficulties in structuring and recognizing the audio content, which may impair subsequent ASR and music understanding applications. This paper proposes a multi-task audio source separation (MTASS) based ASR model called JRSV, which Jointly Recognizes Speech and singing Voices. Specifically, the MTASS module separates the mixed audio into distinct speech and singing voice tracks while removing background music. The CTC/attention hybrid recognition module recognizes both tracks. Online distillation is proposed to improve the robustness of recognition further. To evaluate the proposed methods, a benchmark dataset is constructed and released. Experimental results demonstrate that JRSV can significantly improve recognition accuracy on each track of the mixed audio. |
| title | Jointly Recognizing Speech and Singing Voices Based on Multi-Task Audio Source Separation |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2404.11275 |