CTC-based Non-autoregressive Textless Speech-to-Speech Translation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910482166710272 |
|---|---|
| author | Fang, Qingkai Ma, Zhengrui Zhou, Yan Zhang, Min Feng, Yang |
| author_facet | Fang, Qingkai Ma, Zhengrui Zhou, Yan Zhang, Min Feng, Yang |
| contents | Direct speech-to-speech translation (S2ST) has achieved impressive translation quality, but it often faces the challenge of slow decoding due to the considerable length of speech sequences. Recently, some research has turned to non-autoregressive (NAR) models to expedite decoding, yet the translation quality typically lags behind autoregressive (AR) models significantly. In this paper, we investigate the performance of CTC-based NAR models in S2ST, as these models have shown impressive results in machine translation. Experimental results demonstrate that by combining pretraining, knowledge distillation, and advanced NAR training techniques such as glancing training and non-monotonic latent alignments, CTC-based NAR models achieve translation quality comparable to the AR model, while preserving up to 26.81$\times$ decoding speedup. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07330 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CTC-based Non-autoregressive Textless Speech-to-Speech Translation Fang, Qingkai Ma, Zhengrui Zhou, Yan Zhang, Min Feng, Yang Computation and Language Artificial Intelligence Sound Audio and Speech Processing I.2.7 Direct speech-to-speech translation (S2ST) has achieved impressive translation quality, but it often faces the challenge of slow decoding due to the considerable length of speech sequences. Recently, some research has turned to non-autoregressive (NAR) models to expedite decoding, yet the translation quality typically lags behind autoregressive (AR) models significantly. In this paper, we investigate the performance of CTC-based NAR models in S2ST, as these models have shown impressive results in machine translation. Experimental results demonstrate that by combining pretraining, knowledge distillation, and advanced NAR training techniques such as glancing training and non-monotonic latent alignments, CTC-based NAR models achieve translation quality comparable to the AR model, while preserving up to 26.81$\times$ decoding speedup. |
| title | CTC-based Non-autoregressive Textless Speech-to-Speech Translation |
| topic | Computation and Language Artificial Intelligence Sound Audio and Speech Processing I.2.7 |
| url | https://arxiv.org/abs/2406.07330 |