Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.03988 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913578355785728 |
|---|---|
| author | Tseng, Liang-Hsuan Hu, En-Pei Chiang, Cheng-Han Tseng, Yuan Lee, Hung-yi Lee, Lin-shan Sun, Shao-Hua |
| author_facet | Tseng, Liang-Hsuan Hu, En-Pei Chiang, Cheng-Han Tseng, Yuan Lee, Hung-yi Lee, Lin-shan Sun, Shao-Hua |
| contents | Unsupervised automatic speech recognition (ASR) aims to learn the mapping between the speech signal and its corresponding textual transcription without the supervision of paired speech-text data. A word/phoneme in the speech signal is represented by a segment of speech signal with variable length and unknown boundary, and this segmental structure makes learning the mapping between speech and text challenging, especially without paired data. In this paper, we propose REBORN,Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR. REBORN alternates between (1) training a segmentation model that predicts the boundaries of the segmental structures in speech signals and (2) training the phoneme prediction model, whose input is the speech feature segmented by the segmentation model, to predict a phoneme transcription. Since supervised data for training the segmentation model is not available, we use reinforcement learning to train the segmentation model to favor segmentations that yield phoneme sequence predictions with a lower perplexity. We conduct extensive experiments and find that under the same setting, REBORN outperforms all prior unsupervised ASR models on LibriSpeech, TIMIT, and five non-English languages in Multilingual LibriSpeech. We comprehensively analyze why the boundaries learned by REBORN improve the unsupervised ASR performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_03988 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR Tseng, Liang-Hsuan Hu, En-Pei Chiang, Cheng-Han Tseng, Yuan Lee, Hung-yi Lee, Lin-shan Sun, Shao-Hua Audio and Speech Processing Computation and Language Sound Unsupervised automatic speech recognition (ASR) aims to learn the mapping between the speech signal and its corresponding textual transcription without the supervision of paired speech-text data. A word/phoneme in the speech signal is represented by a segment of speech signal with variable length and unknown boundary, and this segmental structure makes learning the mapping between speech and text challenging, especially without paired data. In this paper, we propose REBORN,Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR. REBORN alternates between (1) training a segmentation model that predicts the boundaries of the segmental structures in speech signals and (2) training the phoneme prediction model, whose input is the speech feature segmented by the segmentation model, to predict a phoneme transcription. Since supervised data for training the segmentation model is not available, we use reinforcement learning to train the segmentation model to favor segmentations that yield phoneme sequence predictions with a lower perplexity. We conduct extensive experiments and find that under the same setting, REBORN outperforms all prior unsupervised ASR models on LibriSpeech, TIMIT, and five non-English languages in Multilingual LibriSpeech. We comprehensively analyze why the boundaries learned by REBORN improve the unsupervised ASR performance. |
| title | REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR |
| topic | Audio and Speech Processing Computation and Language Sound |
| url | https://arxiv.org/abs/2402.03988 |