PURE Codec: Progressive Unfolding of Residual Entropy for Speech Codec Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918221564608512 |
|---|---|
| author | Shi, Jiatong Wang, Haoran Chen, William Li, Chenda Zhang, Wangyou Tian, Jinchuan Watanabe, Shinji |
| author_facet | Shi, Jiatong Wang, Haoran Chen, William Li, Chenda Zhang, Wangyou Tian, Jinchuan Watanabe, Shinji |
| contents | Neural speech codecs have achieved strong performance in low-bitrate compression, but residual vector quantization (RVQ) often suffers from unstable training and ineffective decomposition, limiting reconstruction quality and efficiency. We propose PURE Codec (Progressive Unfolding of Residual Entropy), a novel framework that guides multi-stage quantization using a pre-trained speech enhancement model. The first quantization stage reconstructs low-entropy, denoised speech embeddings, while subsequent stages encode residual high-entropy components. This design improves training stability significantly. Experiments demonstrate that PURE consistently outperforms conventional RVQ-based codecs in reconstruction and downstream speech language model-based text-to-speech, particularly under noisy training conditions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_22687 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PURE Codec: Progressive Unfolding of Residual Entropy for Speech Codec Learning Shi, Jiatong Wang, Haoran Chen, William Li, Chenda Zhang, Wangyou Tian, Jinchuan Watanabe, Shinji Sound Audio and Speech Processing Neural speech codecs have achieved strong performance in low-bitrate compression, but residual vector quantization (RVQ) often suffers from unstable training and ineffective decomposition, limiting reconstruction quality and efficiency. We propose PURE Codec (Progressive Unfolding of Residual Entropy), a novel framework that guides multi-stage quantization using a pre-trained speech enhancement model. The first quantization stage reconstructs low-entropy, denoised speech embeddings, while subsequent stages encode residual high-entropy components. This design improves training stability significantly. Experiments demonstrate that PURE consistently outperforms conventional RVQ-based codecs in reconstruction and downstream speech language model-based text-to-speech, particularly under noisy training conditions. |
| title | PURE Codec: Progressive Unfolding of Residual Entropy for Speech Codec Learning |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2511.22687 |