Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.22220 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911590229475328 |
|---|---|
| author | Song, Yuhan Zhang, Linhao Wu, Chuhan Liu, Aiwei Jia, Wei Wang, Houfeng Zhou, Xiao |
| author_facet | Song, Yuhan Zhang, Linhao Wu, Chuhan Liu, Aiwei Jia, Wei Wang, Houfeng Zhou, Xiao |
| contents | Prevalent semantic speech tokenizers, designed to capture linguistic content, are surprisingly fragile. We find they are not robust to meaning-irrelevant acoustic perturbations; even at high Signal-to-Noise Ratios (SNRs) where speech is perfectly intelligible, their output token sequences can change drastically, increasing the learning burden for downstream LLMs. This instability stems from two flaws: a brittle single-path quantization architecture and a distant training signal indifferent to intermediate token stability. To address this, we introduce StableToken, a tokenizer that achieves stability through a consensus-driven mechanism. Its multi-branch architecture processes audio in parallel, and these representations are merged via a powerful bit-wise voting mechanism to form a single, stable token sequence. StableToken sets a new state-of-the-art in token stability, drastically reducing Unit Edit Distance (UED) under diverse noise conditions. This foundational stability translates directly to downstream benefits, significantly improving the robustness of SpeechLLMs on a variety of tasks. Our code and model are publicly available at https://github.com/Tencent/StableToken. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_22220 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMs Song, Yuhan Zhang, Linhao Wu, Chuhan Liu, Aiwei Jia, Wei Wang, Houfeng Zhou, Xiao Computation and Language Sound Prevalent semantic speech tokenizers, designed to capture linguistic content, are surprisingly fragile. We find they are not robust to meaning-irrelevant acoustic perturbations; even at high Signal-to-Noise Ratios (SNRs) where speech is perfectly intelligible, their output token sequences can change drastically, increasing the learning burden for downstream LLMs. This instability stems from two flaws: a brittle single-path quantization architecture and a distant training signal indifferent to intermediate token stability. To address this, we introduce StableToken, a tokenizer that achieves stability through a consensus-driven mechanism. Its multi-branch architecture processes audio in parallel, and these representations are merged via a powerful bit-wise voting mechanism to form a single, stable token sequence. StableToken sets a new state-of-the-art in token stability, drastically reducing Unit Edit Distance (UED) under diverse noise conditions. This foundational stability translates directly to downstream benefits, significantly improving the robustness of SpeechLLMs on a variety of tasks. Our code and model are publicly available at https://github.com/Tencent/StableToken. |
| title | StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMs |
| topic | Computation and Language Sound |
| url | https://arxiv.org/abs/2509.22220 |