MATE: Matryoshka Audio-Text Embeddings for Open-Vocabulary Keyword Spotting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912835088416768 |
|---|---|
| author | Jung, Youngmoon Jung, Myunghun Yang, Joon-Young Lee, Yong-Hyeok Roh, Jaeyoung Cho, Hoon-Young |
| author_facet | Jung, Youngmoon Jung, Myunghun Yang, Joon-Young Lee, Yong-Hyeok Roh, Jaeyoung Cho, Hoon-Young |
| contents | Open-vocabulary keyword spotting (KWS) with text-based enrollment has emerged as a flexible alternative to fixed-phrase triggers. Prior utterance-level matching methods, from an embedding-learning standpoint, learn embeddings at a single fixed dimensionality. We depart from this design and propose Matryoshka Audio-Text Embeddings (MATE), a dual-encoder framework that encodes multiple embedding granularities within a single vector via nested sub-embeddings ("prefixes"). Specifically, we introduce a PCA-guided prefix alignment: PCA-compressed versions of the full text embedding for each prefix size serve as teacher targets to align both audio and text prefixes. This alignment concentrates salient keyword cues in lower-dimensional prefixes, while higher dimensions add detail. MATE is trained with standard deep metric learning objectives for audio-text KWS, and is loss-agnostic. To our knowledge, this is the first application of matryoshka-style embeddings to KWS, achieving state-of-the-art results on WSJ and LibriPhrase without any inference overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_14012 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MATE: Matryoshka Audio-Text Embeddings for Open-Vocabulary Keyword Spotting Jung, Youngmoon Jung, Myunghun Yang, Joon-Young Lee, Yong-Hyeok Roh, Jaeyoung Cho, Hoon-Young Audio and Speech Processing Artificial Intelligence Open-vocabulary keyword spotting (KWS) with text-based enrollment has emerged as a flexible alternative to fixed-phrase triggers. Prior utterance-level matching methods, from an embedding-learning standpoint, learn embeddings at a single fixed dimensionality. We depart from this design and propose Matryoshka Audio-Text Embeddings (MATE), a dual-encoder framework that encodes multiple embedding granularities within a single vector via nested sub-embeddings ("prefixes"). Specifically, we introduce a PCA-guided prefix alignment: PCA-compressed versions of the full text embedding for each prefix size serve as teacher targets to align both audio and text prefixes. This alignment concentrates salient keyword cues in lower-dimensional prefixes, while higher dimensions add detail. MATE is trained with standard deep metric learning objectives for audio-text KWS, and is loss-agnostic. To our knowledge, this is the first application of matryoshka-style embeddings to KWS, achieving state-of-the-art results on WSJ and LibriPhrase without any inference overhead. |
| title | MATE: Matryoshka Audio-Text Embeddings for Open-Vocabulary Keyword Spotting |
| topic | Audio and Speech Processing Artificial Intelligence |
| url | https://arxiv.org/abs/2601.14012 |