Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.00212 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909124107698176 |
|---|---|
| author | Tathe, Aniket Kamble, Anand Kumbharkar, Suyash Bhandare, Atharva Mitra, Anirban C. |
| author_facet | Tathe, Aniket Kamble, Anand Kumbharkar, Suyash Bhandare, Atharva Mitra, Anirban C. |
| contents | This research addresses the challenge of training an ASR model for personalized voices with minimal data. Utilizing just 14 minutes of custom audio from a YouTube video, we employ Retrieval-Based Voice Conversion (RVC) to create a custom Common Voice 16.0 corpus. Subsequently, a Cross-lingual Self-supervised Representations (XLSR) Wav2Vec2 model is fine-tuned on this dataset. The developed web-based GUI efficiently transcribes and translates input Hindi videos. By integrating XLSR Wav2Vec2 and mBART, the system aligns the translated text with the video timeline, delivering an accessible solution for multilingual video content transcription and translation for personalized voice. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_00212 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Transcription and translation of videos using fine-tuned XLSR Wav2Vec2 on custom dataset and mBART Tathe, Aniket Kamble, Anand Kumbharkar, Suyash Bhandare, Atharva Mitra, Anirban C. Computation and Language Computer Vision and Pattern Recognition Machine Learning Sound Audio and Speech Processing This research addresses the challenge of training an ASR model for personalized voices with minimal data. Utilizing just 14 minutes of custom audio from a YouTube video, we employ Retrieval-Based Voice Conversion (RVC) to create a custom Common Voice 16.0 corpus. Subsequently, a Cross-lingual Self-supervised Representations (XLSR) Wav2Vec2 model is fine-tuned on this dataset. The developed web-based GUI efficiently transcribes and translates input Hindi videos. By integrating XLSR Wav2Vec2 and mBART, the system aligns the translated text with the video timeline, delivering an accessible solution for multilingual video content transcription and translation for personalized voice. |
| title | Transcription and translation of videos using fine-tuned XLSR Wav2Vec2 on custom dataset and mBART |
| topic | Computation and Language Computer Vision and Pattern Recognition Machine Learning Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2403.00212 |