End-to-End Evaluation for Low-Latency Simultaneous Speech Translation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911039957762048 |
|---|---|
| author | Huber, Christian Dinh, Tu Anh Mullov, Carlos Pham, Ngoc Quan Nguyen, Thai Binh Retkowski, Fabian Constantin, Stefan Ugan, Enes Yavuz Liu, Danni Li, Zhaolin Koneru, Sai Niehues, Jan Waibel, Alexander |
| author_facet | Huber, Christian Dinh, Tu Anh Mullov, Carlos Pham, Ngoc Quan Nguyen, Thai Binh Retkowski, Fabian Constantin, Stefan Ugan, Enes Yavuz Liu, Danni Li, Zhaolin Koneru, Sai Niehues, Jan Waibel, Alexander |
| contents | The challenge of low-latency speech translation has recently draw significant interest in the research community as shown by several publications and shared tasks. Therefore, it is essential to evaluate these different approaches in realistic scenarios. However, currently only specific aspects of the systems are evaluated and often it is not possible to compare different approaches.
In this work, we propose the first framework to perform and evaluate the various aspects of low-latency speech translation under realistic conditions. The evaluation is carried out in an end-to-end fashion. This includes the segmentation of the audio as well as the run-time of the different components.
Secondly, we compare different approaches to low-latency speech translation using this framework. We evaluate models with the option to revise the output as well as methods with fixed output. Furthermore, we directly compare state-of-the-art cascaded as well as end-to-end systems. Finally, the framework allows to automatically evaluate the translation quality as well as latency and also provides a web interface to show the low-latency model outputs to the user. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_03415 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | End-to-End Evaluation for Low-Latency Simultaneous Speech Translation Huber, Christian Dinh, Tu Anh Mullov, Carlos Pham, Ngoc Quan Nguyen, Thai Binh Retkowski, Fabian Constantin, Stefan Ugan, Enes Yavuz Liu, Danni Li, Zhaolin Koneru, Sai Niehues, Jan Waibel, Alexander Computation and Language Artificial Intelligence The challenge of low-latency speech translation has recently draw significant interest in the research community as shown by several publications and shared tasks. Therefore, it is essential to evaluate these different approaches in realistic scenarios. However, currently only specific aspects of the systems are evaluated and often it is not possible to compare different approaches. In this work, we propose the first framework to perform and evaluate the various aspects of low-latency speech translation under realistic conditions. The evaluation is carried out in an end-to-end fashion. This includes the segmentation of the audio as well as the run-time of the different components. Secondly, we compare different approaches to low-latency speech translation using this framework. We evaluate models with the option to revise the output as well as methods with fixed output. Furthermore, we directly compare state-of-the-art cascaded as well as end-to-end systems. Finally, the framework allows to automatically evaluate the translation quality as well as latency and also provides a web interface to show the low-latency model outputs to the user. |
| title | End-to-End Evaluation for Low-Latency Simultaneous Speech Translation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2308.03415 |