End-to-end Joint Punctuated and Normalized ASR with a Limited Amount of Punctuated Training Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908456199389184 |
|---|---|
| author | Cui, Can Sheikh, Imran Ahamad Sadeghi, Mostafa Vincent, Emmanuel |
| author_facet | Cui, Can Sheikh, Imran Ahamad Sadeghi, Mostafa Vincent, Emmanuel |
| contents | Joint punctuated and normalized automatic speech recognition (ASR) aims at outputing transcripts with and without punctuation and casing. This task remains challenging due to the lack of paired speech and punctuated text data in most ASR corpora. We propose two approaches to train an end-to-end joint punctuated and normalized ASR system using limited punctuated data. The first approach uses a language model to convert normalized training transcripts into punctuated transcripts. This achieves a better performance on out-of-domain test data, with up to 17% relative Punctuation-Case-aware Word Error Rate (PC-WER) reduction. The second approach uses a single decoder conditioned on the type of output. This yields a 42% relative PC-WER reduction compared to Whisper-base and a 4% relative (normalized) WER reduction compared to the normalized output of a punctuated-only model. Additionally, our proposed model demonstrates the feasibility of a joint ASR system using as little as 5% punctuated training data with a moderate (2.42% absolute) PC-WER increase. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_17741 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | End-to-end Joint Punctuated and Normalized ASR with a Limited Amount of Punctuated Training Data Cui, Can Sheikh, Imran Ahamad Sadeghi, Mostafa Vincent, Emmanuel Computation and Language Sound Audio and Speech Processing Joint punctuated and normalized automatic speech recognition (ASR) aims at outputing transcripts with and without punctuation and casing. This task remains challenging due to the lack of paired speech and punctuated text data in most ASR corpora. We propose two approaches to train an end-to-end joint punctuated and normalized ASR system using limited punctuated data. The first approach uses a language model to convert normalized training transcripts into punctuated transcripts. This achieves a better performance on out-of-domain test data, with up to 17% relative Punctuation-Case-aware Word Error Rate (PC-WER) reduction. The second approach uses a single decoder conditioned on the type of output. This yields a 42% relative PC-WER reduction compared to Whisper-base and a 4% relative (normalized) WER reduction compared to the normalized output of a punctuated-only model. Additionally, our proposed model demonstrates the feasibility of a joint ASR system using as little as 5% punctuated training data with a moderate (2.42% absolute) PC-WER increase. |
| title | End-to-end Joint Punctuated and Normalized ASR with a Limited Amount of Punctuated Training Data |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2311.17741 |