Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.15726 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912339860652032 |
|---|---|
| author | Timmel, Vincenzo Paonessa, Claudio Kakooee, Reza Vogel, Manfred Perruchoud, Daniel |
| author_facet | Timmel, Vincenzo Paonessa, Claudio Kakooee, Reza Vogel, Manfred Perruchoud, Daniel |
| contents | This paper presents a new approach to fine-tuning OpenAI's Whisper model for low-resource languages by introducing a novel data generation method that converts sentence-level data into a long-form corpus, using Swiss German as a case study. Non-sentence-level data, which could improve the performance of long-form audio, is difficult to obtain and often restricted by copyright laws. Our method bridges this gap by transforming more accessible sentence-level data into a format that preserves the model's ability to handle long-form audio and perform segmentation without requiring non-sentence-level data. Our data generation process improves performance in several real-world applications and leads to the development of a new state-of-the-art speech-to-text (STT) model for Swiss German. We compare our model with a non-fine-tuned Whisper and our previous state-of-the-art Swiss German STT models, where our new model achieves higher BLEU scores. Our results also indicate that the proposed method is adaptable to other low-resource languages, supported by written guidance and code that allows the creation of fine-tuned Whisper models, which keep segmentation capabilities and allow the transcription of longer audio files using only sentence-level data with high quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15726 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Fine-tuning Whisper on Low-Resource Languages for Real-World Applications Timmel, Vincenzo Paonessa, Claudio Kakooee, Reza Vogel, Manfred Perruchoud, Daniel Computation and Language Audio and Speech Processing This paper presents a new approach to fine-tuning OpenAI's Whisper model for low-resource languages by introducing a novel data generation method that converts sentence-level data into a long-form corpus, using Swiss German as a case study. Non-sentence-level data, which could improve the performance of long-form audio, is difficult to obtain and often restricted by copyright laws. Our method bridges this gap by transforming more accessible sentence-level data into a format that preserves the model's ability to handle long-form audio and perform segmentation without requiring non-sentence-level data. Our data generation process improves performance in several real-world applications and leads to the development of a new state-of-the-art speech-to-text (STT) model for Swiss German. We compare our model with a non-fine-tuned Whisper and our previous state-of-the-art Swiss German STT models, where our new model achieves higher BLEU scores. Our results also indicate that the proposed method is adaptable to other low-resource languages, supported by written guidance and code that allows the creation of fine-tuned Whisper models, which keep segmentation capabilities and allow the transcription of longer audio files using only sentence-level data with high quality. |
| title | Fine-tuning Whisper on Low-Resource Languages for Real-World Applications |
| topic | Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2412.15726 |