Bilevel Joint Unsupervised and Supervised Training for Automatic Speech Recognition
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916518761070592 |
|---|---|
| author | Cui, Xiaodong Saif, A F M Lu, Songtao Chen, Lisha Chen, Tianyi Kingsbury, Brian Saon, George |
| author_facet | Cui, Xiaodong Saif, A F M Lu, Songtao Chen, Lisha Chen, Tianyi Kingsbury, Brian Saon, George |
| contents | In this paper, we propose a bilevel joint unsupervised and supervised training (BL-JUST) framework for automatic speech recognition. Compared to the conventional pre-training and fine-tuning strategy which is a disconnected two-stage process, BL-JUST tries to optimize an acoustic model such that it simultaneously minimizes both the unsupervised and supervised loss functions. Because BL-JUST seeks matched local optima of both loss functions, acoustic representations learned by the acoustic model strike a good balance between being generic and task-specific. We solve the BL-JUST problem using penalty-based bilevel gradient descent and evaluate the trained deep neural network acoustic models on various datasets with a variety of architectures and loss functions. We show that BL-JUST can outperform the widely-used pre-training and fine-tuning strategy and some other popular semi-supervised techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_08548 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Bilevel Joint Unsupervised and Supervised Training for Automatic Speech Recognition Cui, Xiaodong Saif, A F M Lu, Songtao Chen, Lisha Chen, Tianyi Kingsbury, Brian Saon, George Computation and Language In this paper, we propose a bilevel joint unsupervised and supervised training (BL-JUST) framework for automatic speech recognition. Compared to the conventional pre-training and fine-tuning strategy which is a disconnected two-stage process, BL-JUST tries to optimize an acoustic model such that it simultaneously minimizes both the unsupervised and supervised loss functions. Because BL-JUST seeks matched local optima of both loss functions, acoustic representations learned by the acoustic model strike a good balance between being generic and task-specific. We solve the BL-JUST problem using penalty-based bilevel gradient descent and evaluate the trained deep neural network acoustic models on various datasets with a variety of architectures and loss functions. We show that BL-JUST can outperform the widely-used pre-training and fine-tuning strategy and some other popular semi-supervised techniques. |
| title | Bilevel Joint Unsupervised and Supervised Training for Automatic Speech Recognition |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.08548 |