NAIST Simultaneous Speech Translation System for IWSLT 2024
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929404578365440 |
|---|---|
| author | Ko, Yuka Fukuda, Ryo Nishikawa, Yuta Kano, Yasumasa Yanagita, Tomoya Doi, Kosuke Makinae, Mana Tan, Haotian Sakai, Makoto Sakti, Sakriani Sudoh, Katsuhito Nakamura, Satoshi |
| author_facet | Ko, Yuka Fukuda, Ryo Nishikawa, Yuta Kano, Yasumasa Yanagita, Tomoya Doi, Kosuke Makinae, Mana Tan, Haotian Sakai, Makoto Sakti, Sakriani Sudoh, Katsuhito Nakamura, Satoshi |
| contents | This paper describes NAIST's submission to the simultaneous track of the IWSLT 2024 Evaluation Campaign: English-to-{German, Japanese, Chinese} speech-to-text translation and English-to-Japanese speech-to-speech translation. We develop a multilingual end-to-end speech-to-text translation model combining two pre-trained language models, HuBERT and mBART. We trained this model with two decoding policies, Local Agreement (LA) and AlignAtt. The submitted models employ the LA policy because it outperformed the AlignAtt policy in previous models. Our speech-to-speech translation method is a cascade of the above speech-to-text model and an incremental text-to-speech (TTS) module that incorporates a phoneme estimation model, a parallel acoustic model, and a parallel WaveGAN vocoder. We improved our incremental TTS by applying the Transformer architecture with the AlignAtt policy for the estimation model. The results show that our upgraded TTS module contributed to improving the system performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_00826 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | NAIST Simultaneous Speech Translation System for IWSLT 2024 Ko, Yuka Fukuda, Ryo Nishikawa, Yuta Kano, Yasumasa Yanagita, Tomoya Doi, Kosuke Makinae, Mana Tan, Haotian Sakai, Makoto Sakti, Sakriani Sudoh, Katsuhito Nakamura, Satoshi Computation and Language Sound Audio and Speech Processing This paper describes NAIST's submission to the simultaneous track of the IWSLT 2024 Evaluation Campaign: English-to-{German, Japanese, Chinese} speech-to-text translation and English-to-Japanese speech-to-speech translation. We develop a multilingual end-to-end speech-to-text translation model combining two pre-trained language models, HuBERT and mBART. We trained this model with two decoding policies, Local Agreement (LA) and AlignAtt. The submitted models employ the LA policy because it outperformed the AlignAtt policy in previous models. Our speech-to-speech translation method is a cascade of the above speech-to-text model and an incremental text-to-speech (TTS) module that incorporates a phoneme estimation model, a parallel acoustic model, and a parallel WaveGAN vocoder. We improved our incremental TTS by applying the Transformer architecture with the AlignAtt policy for the estimation model. The results show that our upgraded TTS module contributed to improving the system performance. |
| title | NAIST Simultaneous Speech Translation System for IWSLT 2024 |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2407.00826 |