Whisper in Medusa's Ear: Multi-head Efficient Decoding for Transformer-based ASR
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909324091064320 |
|---|---|
| author | Segal-Feldman, Yael Shamsian, Aviv Navon, Aviv Hetz, Gill Keshet, Joseph |
| author_facet | Segal-Feldman, Yael Shamsian, Aviv Navon, Aviv Hetz, Gill Keshet, Joseph |
| contents | Large transformer-based models have significant potential for speech transcription and translation. Their self-attention mechanisms and parallel processing enable them to capture complex patterns and dependencies in audio sequences. However, this potential comes with challenges, as these large and computationally intensive models lead to slow inference speeds. Various optimization strategies have been proposed to improve performance, including efficient hardware utilization and algorithmic enhancements. In this paper, we introduce Whisper-Medusa, a novel approach designed to enhance processing speed with minimal impact on Word Error Rate (WER). The proposed model extends the OpenAI's Whisper architecture by predicting multiple tokens per iteration, resulting in a 50% reduction in latency. We showcase the effectiveness of Whisper-Medusa across different learning setups and datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15869 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Whisper in Medusa's Ear: Multi-head Efficient Decoding for Transformer-based ASR Segal-Feldman, Yael Shamsian, Aviv Navon, Aviv Hetz, Gill Keshet, Joseph Audio and Speech Processing Artificial Intelligence Machine Learning Sound Large transformer-based models have significant potential for speech transcription and translation. Their self-attention mechanisms and parallel processing enable them to capture complex patterns and dependencies in audio sequences. However, this potential comes with challenges, as these large and computationally intensive models lead to slow inference speeds. Various optimization strategies have been proposed to improve performance, including efficient hardware utilization and algorithmic enhancements. In this paper, we introduce Whisper-Medusa, a novel approach designed to enhance processing speed with minimal impact on Word Error Rate (WER). The proposed model extends the OpenAI's Whisper architecture by predicting multiple tokens per iteration, resulting in a 50% reduction in latency. We showcase the effectiveness of Whisper-Medusa across different learning setups and datasets. |
| title | Whisper in Medusa's Ear: Multi-head Efficient Decoding for Transformer-based ASR |
| topic | Audio and Speech Processing Artificial Intelligence Machine Learning Sound |
| url | https://arxiv.org/abs/2409.15869 |