Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916562454183936 |
|---|---|
| author | Wang, Bin Zou, Xunlong Sun, Shuo Zhang, Wenyu He, Yingxu Liu, Zhuohan Wei, Chengwei Chen, Nancy F. Aw, AiTi |
| author_facet | Wang, Bin Zou, Xunlong Sun, Shuo Zhang, Wenyu He, Yingxu Liu, Zhuohan Wei, Chengwei Chen, Nancy F. Aw, AiTi |
| contents | Singlish, a Creole language rooted in English, is a key focus in linguistic research within multilingual and multicultural contexts. However, its spoken form remains underexplored, limiting insights into its linguistic structure and applications. To address this gap, we standardize and annotate the largest spoken Singlish corpus, introducing the Multitask National Speech Corpus (MNSC). These datasets support diverse tasks, including Automatic Speech Recognition (ASR), Spoken Question Answering (SQA), Spoken Dialogue Summarization (SDS), and Paralinguistic Question Answering (PQA). We release standardized splits and a human-verified test set to facilitate further research. Additionally, we propose SingAudioLLM, a multi-task multimodal model leveraging multimodal large language models to handle these tasks concurrently. Experiments reveal our models adaptability to Singlish context, achieving state-of-the-art performance and outperforming prior models by 10-30% in comparison with other AudioLLMs and cascaded solutions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_01034 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Models Wang, Bin Zou, Xunlong Sun, Shuo Zhang, Wenyu He, Yingxu Liu, Zhuohan Wei, Chengwei Chen, Nancy F. Aw, AiTi Computation and Language Sound Audio and Speech Processing Singlish, a Creole language rooted in English, is a key focus in linguistic research within multilingual and multicultural contexts. However, its spoken form remains underexplored, limiting insights into its linguistic structure and applications. To address this gap, we standardize and annotate the largest spoken Singlish corpus, introducing the Multitask National Speech Corpus (MNSC). These datasets support diverse tasks, including Automatic Speech Recognition (ASR), Spoken Question Answering (SQA), Spoken Dialogue Summarization (SDS), and Paralinguistic Question Answering (PQA). We release standardized splits and a human-verified test set to facilitate further research. Additionally, we propose SingAudioLLM, a multi-task multimodal model leveraging multimodal large language models to handle these tasks concurrently. Experiments reveal our models adaptability to Singlish context, achieving state-of-the-art performance and outperforming prior models by 10-30% in comparison with other AudioLLMs and cascaded solutions. |
| title | Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Models |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2501.01034 |