Decoder-only Architecture for Streaming End-to-end Speech Recognition
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909275880685568 |
|---|---|
| author | Tsunoo, Emiru Futami, Hayato Kashiwagi, Yosuke Arora, Siddhant Watanabe, Shinji |
| author_facet | Tsunoo, Emiru Futami, Hayato Kashiwagi, Yosuke Arora, Siddhant Watanabe, Shinji |
| contents | Decoder-only language models (LMs) have been successfully adopted for speech-processing tasks including automatic speech recognition (ASR). The LMs have ample expressiveness and perform efficiently. This efficiency is a suitable characteristic for streaming applications of ASR. In this work, we propose to use a decoder-only architecture for blockwise streaming ASR. In our approach, speech features are compressed using CTC output and context embedding using blockwise speech subnetwork, and are sequentially provided as prompts to the decoder. The decoder estimates the output tokens promptly at each block. To this end, we also propose a novel training scheme using random-length prefix prompts to make the model robust to the truncated prompts caused by blockwise processing. An experimental comparison shows that our proposed decoder-only streaming ASR achieves 8% relative word error rate reduction in the LibriSpeech test-other set while being twice as fast as the baseline model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_16107 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Decoder-only Architecture for Streaming End-to-end Speech Recognition Tsunoo, Emiru Futami, Hayato Kashiwagi, Yosuke Arora, Siddhant Watanabe, Shinji Audio and Speech Processing Computation and Language Decoder-only language models (LMs) have been successfully adopted for speech-processing tasks including automatic speech recognition (ASR). The LMs have ample expressiveness and perform efficiently. This efficiency is a suitable characteristic for streaming applications of ASR. In this work, we propose to use a decoder-only architecture for blockwise streaming ASR. In our approach, speech features are compressed using CTC output and context embedding using blockwise speech subnetwork, and are sequentially provided as prompts to the decoder. The decoder estimates the output tokens promptly at each block. To this end, we also propose a novel training scheme using random-length prefix prompts to make the model robust to the truncated prompts caused by blockwise processing. An experimental comparison shows that our proposed decoder-only streaming ASR achieves 8% relative word error rate reduction in the LibriSpeech test-other set while being twice as fast as the baseline model. |
| title | Decoder-only Architecture for Streaming End-to-end Speech Recognition |
| topic | Audio and Speech Processing Computation and Language |
| url | https://arxiv.org/abs/2406.16107 |