OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916370402246656 |
|---|---|
| author | Peng, Yifan Tian, Jinchuan Chen, William Arora, Siddhant Yan, Brian Sudo, Yui Shakeel, Muhammad Choi, Kwanghee Shi, Jiatong Chang, Xuankai Jung, Jee-weon Watanabe, Shinji |
| author_facet | Peng, Yifan Tian, Jinchuan Chen, William Arora, Siddhant Yan, Brian Sudo, Yui Shakeel, Muhammad Choi, Kwanghee Shi, Jiatong Chang, Xuankai Jung, Jee-weon Watanabe, Shinji |
| contents | Recent studies have highlighted the importance of fully open foundation models. The Open Whisper-style Speech Model (OWSM) is an initial step towards reproducing OpenAI Whisper using public data and open-source toolkits. However, previous versions of OWSM (v1 to v3) are still based on standard Transformer, which might lead to inferior performance compared to state-of-the-art speech encoder architectures. This work aims to improve the performance and efficiency of OWSM without additional data. We present a series of E-Branchformer-based models named OWSM v3.1, ranging from 100M to 1B parameters. OWSM v3.1 outperforms its predecessor, OWSM v3, in most evaluation benchmarks, while showing an improved inference speed of up to 25%. We further reveal the emergent ability of OWSM v3.1 in zero-shot contextual biasing speech recognition. We also provide a model trained on a subset of data with low license restrictions. We will publicly release the code, pre-trained models, and training logs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_16658 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer Peng, Yifan Tian, Jinchuan Chen, William Arora, Siddhant Yan, Brian Sudo, Yui Shakeel, Muhammad Choi, Kwanghee Shi, Jiatong Chang, Xuankai Jung, Jee-weon Watanabe, Shinji Computation and Language Audio and Speech Processing Recent studies have highlighted the importance of fully open foundation models. The Open Whisper-style Speech Model (OWSM) is an initial step towards reproducing OpenAI Whisper using public data and open-source toolkits. However, previous versions of OWSM (v1 to v3) are still based on standard Transformer, which might lead to inferior performance compared to state-of-the-art speech encoder architectures. This work aims to improve the performance and efficiency of OWSM without additional data. We present a series of E-Branchformer-based models named OWSM v3.1, ranging from 100M to 1B parameters. OWSM v3.1 outperforms its predecessor, OWSM v3, in most evaluation benchmarks, while showing an improved inference speed of up to 25%. We further reveal the emergent ability of OWSM v3.1 in zero-shot contextual biasing speech recognition. We also provide a model trained on a subset of data with low license restrictions. We will publicly release the code, pre-trained models, and training logs. |
| title | OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer |
| topic | Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2401.16658 |