Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2511.04139 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909889871216640 |
|---|---|
| author | Chen, Dazhong Lin, Yi-Cheng Huang, Yuchen Gong, Ziwei Jiang, Di Xie, Zeying R., Yi Fung |
| author_facet | Chen, Dazhong Lin, Yi-Cheng Huang, Yuchen Gong, Ziwei Jiang, Di Xie, Zeying R., Yi Fung |
| contents | Automatic speech recognition (ASR) is critical for language accessibility, yet low-resource Cantonese remains challenging due to limited annotated data, six lexical tones, tone sandhi, and accent variation. Existing ASR models, such as Whisper, often suffer from high word error rates. Large audio-language models (LALMs), in contrast, can leverage broader contextual reasoning but still require explicit tonal and prosodic acoustic cues. We introduce CantoASR, a collaborative ASR-LALM error correction framework that integrates forced alignment for acoustic feature extraction, a LoRA-finetuned Whisper for improved tone discrimination, and an instruction-tuned Qwen-Audio for prosody-aware correction. Evaluations on spontaneous Cantonese data show substantial CER gains over Whisper-Large-V3. These findings suggest that integrating acoustic cues with LALM reasoning provides a scalable strategy for low-resource tonal and dialectal ASR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_04139 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonese Chen, Dazhong Lin, Yi-Cheng Huang, Yuchen Gong, Ziwei Jiang, Di Xie, Zeying R., Yi Fung Computation and Language Sound Automatic speech recognition (ASR) is critical for language accessibility, yet low-resource Cantonese remains challenging due to limited annotated data, six lexical tones, tone sandhi, and accent variation. Existing ASR models, such as Whisper, often suffer from high word error rates. Large audio-language models (LALMs), in contrast, can leverage broader contextual reasoning but still require explicit tonal and prosodic acoustic cues. We introduce CantoASR, a collaborative ASR-LALM error correction framework that integrates forced alignment for acoustic feature extraction, a LoRA-finetuned Whisper for improved tone discrimination, and an instruction-tuned Qwen-Audio for prosody-aware correction. Evaluations on spontaneous Cantonese data show substantial CER gains over Whisper-Large-V3. These findings suggest that integrating acoustic cues with LALM reasoning provides a scalable strategy for low-resource tonal and dialectal ASR. |
| title | CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonese |
| topic | Computation and Language Sound |
| url | https://arxiv.org/abs/2511.04139 |