Whisper based Cross-Lingual Phoneme Recognition between Vietnamese and English
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918131168968704 |
|---|---|
| author | Minh, Nguyen Huu Nhat Anh, Tran Nguyen Dung, Truong Dinh Van Nam, Vo Tuyen, Le Pham |
| author_facet | Minh, Nguyen Huu Nhat Anh, Tran Nguyen Dung, Truong Dinh Van Nam, Vo Tuyen, Le Pham |
| contents | Cross-lingual phoneme recognition has emerged as a significant challenge for accurate automatic speech recognition (ASR) when mixing Vietnamese and English pronunciations. Unlike many languages, Vietnamese relies on tonal variations to distinguish word meanings, whereas English features stress patterns and non-standard pronunciations that hinder phoneme alignment between the two languages. To address this challenge, we propose a novel bilingual speech recognition approach with two primary contributions: (1) constructing a representative bilingual phoneme set that bridges the differences between Vietnamese and English phonetic systems; (2) designing an end-to-end system that leverages the PhoWhisper pre-trained encoder for deep high-level representations to improve phoneme recognition. Our extensive experiments demonstrate that the proposed approach not only improves recognition accuracy in bilingual speech recognition for Vietnamese but also provides a robust framework for addressing the complexities of tonal and stress-based phoneme recognition |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_19270 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Whisper based Cross-Lingual Phoneme Recognition between Vietnamese and English Minh, Nguyen Huu Nhat Anh, Tran Nguyen Dung, Truong Dinh Van Nam, Vo Tuyen, Le Pham Computation and Language Artificial Intelligence Cross-lingual phoneme recognition has emerged as a significant challenge for accurate automatic speech recognition (ASR) when mixing Vietnamese and English pronunciations. Unlike many languages, Vietnamese relies on tonal variations to distinguish word meanings, whereas English features stress patterns and non-standard pronunciations that hinder phoneme alignment between the two languages. To address this challenge, we propose a novel bilingual speech recognition approach with two primary contributions: (1) constructing a representative bilingual phoneme set that bridges the differences between Vietnamese and English phonetic systems; (2) designing an end-to-end system that leverages the PhoWhisper pre-trained encoder for deep high-level representations to improve phoneme recognition. Our extensive experiments demonstrate that the proposed approach not only improves recognition accuracy in bilingual speech recognition for Vietnamese but also provides a robust framework for addressing the complexities of tonal and stress-based phoneme recognition |
| title | Whisper based Cross-Lingual Phoneme Recognition between Vietnamese and English |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2508.19270 |