Text-Aware Adapter for Few-Shot Keyword Spotting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913859031269376 |
|---|---|
| author | Jung, Youngmoon Lee, Jinyoung Lee, Seungjin Jung, Myunghun Lee, Yong-Hyeok Cho, Hoon-Young |
| author_facet | Jung, Youngmoon Lee, Jinyoung Lee, Seungjin Jung, Myunghun Lee, Yong-Hyeok Cho, Hoon-Young |
| contents | Recent advances in flexible keyword spotting (KWS) with text enrollment allow users to personalize keywords without uttering them during enrollment. However, there is still room for improvement in target keyword performance. In this work, we propose a novel few-shot transfer learning method, called text-aware adapter (TA-adapter), designed to enhance a pre-trained flexible KWS model for specific keywords with limited speech samples. To adapt the acoustic encoder, we leverage a jointly pre-trained text encoder to generate a text embedding that acts as a representative vector for the keyword. By fine-tuning only a small portion of the network while keeping the core components' weights intact, the TA-adapter proves highly efficient for few-shot KWS, enabling a seamless return to the original pre-trained model. In our experiments, the TA-adapter demonstrated significant performance improvements across 35 distinct keywords from the Google Speech Commands V2 dataset, with only a 0.14% increase in the total number of parameters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_18142 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Text-Aware Adapter for Few-Shot Keyword Spotting Jung, Youngmoon Lee, Jinyoung Lee, Seungjin Jung, Myunghun Lee, Yong-Hyeok Cho, Hoon-Young Audio and Speech Processing Artificial Intelligence Signal Processing Recent advances in flexible keyword spotting (KWS) with text enrollment allow users to personalize keywords without uttering them during enrollment. However, there is still room for improvement in target keyword performance. In this work, we propose a novel few-shot transfer learning method, called text-aware adapter (TA-adapter), designed to enhance a pre-trained flexible KWS model for specific keywords with limited speech samples. To adapt the acoustic encoder, we leverage a jointly pre-trained text encoder to generate a text embedding that acts as a representative vector for the keyword. By fine-tuning only a small portion of the network while keeping the core components' weights intact, the TA-adapter proves highly efficient for few-shot KWS, enabling a seamless return to the original pre-trained model. In our experiments, the TA-adapter demonstrated significant performance improvements across 35 distinct keywords from the Google Speech Commands V2 dataset, with only a 0.14% increase in the total number of parameters. |
| title | Text-Aware Adapter for Few-Shot Keyword Spotting |
| topic | Audio and Speech Processing Artificial Intelligence Signal Processing |
| url | https://arxiv.org/abs/2412.18142 |