ViSpeR: Multilingual Audio-Visual Speech Recognition
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914818928148480 |
|---|---|
| author | Narayan, Sanath Djilali, Yasser Abdelaziz Dahou Singh, Ankit Bihan, Eustache Le Hacid, Hakim |
| author_facet | Narayan, Sanath Djilali, Yasser Abdelaziz Dahou Singh, Ankit Bihan, Eustache Le Hacid, Hakim |
| contents | This work presents an extensive and detailed study on Audio-Visual Speech Recognition (AVSR) for five widely spoken languages: Chinese, Spanish, English, Arabic, and French. We have collected large-scale datasets for each language except for English, and have engaged in the training of supervised learning models. Our model, ViSpeR, is trained in a multi-lingual setting, resulting in competitive performance on newly established benchmarks for each language. The datasets and models are released to the community with an aim to serve as a foundation for triggering and feeding further research work and exploration on Audio-Visual Speech Recognition, an increasingly important area of research. Code available at \href{https://github.com/YasserdahouML/visper}{https://github.com/YasserdahouML/visper}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_00038 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ViSpeR: Multilingual Audio-Visual Speech Recognition Narayan, Sanath Djilali, Yasser Abdelaziz Dahou Singh, Ankit Bihan, Eustache Le Hacid, Hakim Computation and Language Artificial Intelligence This work presents an extensive and detailed study on Audio-Visual Speech Recognition (AVSR) for five widely spoken languages: Chinese, Spanish, English, Arabic, and French. We have collected large-scale datasets for each language except for English, and have engaged in the training of supervised learning models. Our model, ViSpeR, is trained in a multi-lingual setting, resulting in competitive performance on newly established benchmarks for each language. The datasets and models are released to the community with an aim to serve as a foundation for triggering and feeding further research work and exploration on Audio-Visual Speech Recognition, an increasingly important area of research. Code available at \href{https://github.com/YasserdahouML/visper}{https://github.com/YasserdahouML/visper}. |
| title | ViSpeR: Multilingual Audio-Visual Speech Recognition |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2406.00038 |