Promptformer: Prompted Conformer Transducer for ASR
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909073176264704 |
|---|---|
| author | Duarte-Torres, Sergio Sen, Arunasish Rana, Aman Drude, Lukas Gomez-Alanis, Alejandro Schwarz, Andreas Rädel, Leif Leutnant, Volker |
| author_facet | Duarte-Torres, Sergio Sen, Arunasish Rana, Aman Drude, Lukas Gomez-Alanis, Alejandro Schwarz, Andreas Rädel, Leif Leutnant, Volker |
| contents | Context cues carry information which can improve multi-turn interactions in automatic speech recognition (ASR) systems. In this paper, we introduce a novel mechanism inspired by hyper-prompting to fuse textual context with acoustic representations in the attention mechanism. Results on a test set with multi-turn interactions show that our method achieves 5.9% relative word error rate reduction (rWERR) over a strong baseline. We show that our method does not degrade in the absence of context and leads to improvements even if the model is trained without context. We further show that leveraging a pre-trained sentence-piece model for context embedding generation can outperform an external BERT model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_07360 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Promptformer: Prompted Conformer Transducer for ASR Duarte-Torres, Sergio Sen, Arunasish Rana, Aman Drude, Lukas Gomez-Alanis, Alejandro Schwarz, Andreas Rädel, Leif Leutnant, Volker Computation and Language Sound Audio and Speech Processing Context cues carry information which can improve multi-turn interactions in automatic speech recognition (ASR) systems. In this paper, we introduce a novel mechanism inspired by hyper-prompting to fuse textual context with acoustic representations in the attention mechanism. Results on a test set with multi-turn interactions show that our method achieves 5.9% relative word error rate reduction (rWERR) over a strong baseline. We show that our method does not degrade in the absence of context and leads to improvements even if the model is trained without context. We further show that leveraging a pre-trained sentence-piece model for context embedding generation can outperform an external BERT model. |
| title | Promptformer: Prompted Conformer Transducer for ASR |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2401.07360 |