CTC-aligned Audio-Text Embedding for Streaming Open-vocabulary Keyword Spotting
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909326857207808 |
|---|---|
| author | Jin, Sichen Jung, Youngmoon Lee, Seungjin Roh, Jaeyoung Han, Changwoo Cho, Hoonyoung |
| author_facet | Jin, Sichen Jung, Youngmoon Lee, Seungjin Roh, Jaeyoung Han, Changwoo Cho, Hoonyoung |
| contents | This paper introduces a novel approach for streaming openvocabulary keyword spotting (KWS) with text-based keyword enrollment. For every input frame, the proposed method finds the optimal alignment ending at the frame using connectionist temporal classification (CTC) and aggregates the frame-level acoustic embedding (AE) to obtain higher-level (i.e., character, word, or phrase) AE that aligns with the text embedding (TE) of the target keyword text. After that, we calculate the similarity of the aggregated AE and the TE. To the best of our knowledge, this is the first attempt to dynamically align the audio and the keyword text on-the-fly to attain the joint audio-text embedding for KWS. Despite operating in a streaming fashion, our approach achieves competitive performance on the LibriPhrase dataset compared to the non-streaming methods with a mere 155K model parameters and a decoding algorithm with time complexity O(U), where U is the length of the target keyword at inference time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07923 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CTC-aligned Audio-Text Embedding for Streaming Open-vocabulary Keyword Spotting Jin, Sichen Jung, Youngmoon Lee, Seungjin Roh, Jaeyoung Han, Changwoo Cho, Hoonyoung Sound Artificial Intelligence Audio and Speech Processing This paper introduces a novel approach for streaming openvocabulary keyword spotting (KWS) with text-based keyword enrollment. For every input frame, the proposed method finds the optimal alignment ending at the frame using connectionist temporal classification (CTC) and aggregates the frame-level acoustic embedding (AE) to obtain higher-level (i.e., character, word, or phrase) AE that aligns with the text embedding (TE) of the target keyword text. After that, we calculate the similarity of the aggregated AE and the TE. To the best of our knowledge, this is the first attempt to dynamically align the audio and the keyword text on-the-fly to attain the joint audio-text embedding for KWS. Despite operating in a streaming fashion, our approach achieves competitive performance on the LibriPhrase dataset compared to the non-streaming methods with a mere 155K model parameters and a decoding algorithm with time complexity O(U), where U is the length of the target keyword at inference time. |
| title | CTC-aligned Audio-Text Embedding for Streaming Open-vocabulary Keyword Spotting |
| topic | Sound Artificial Intelligence Audio and Speech Processing |
| url | https://arxiv.org/abs/2406.07923 |