Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.21224 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914413614727168 |
|---|---|
| author | Zhou, Haoguang Wang, Siyi Wu, Jingyao Bailey, James Dang, Ting |
| author_facet | Zhou, Haoguang Wang, Siyi Wu, Jingyao Bailey, James Dang, Ting |
| contents | Modern speech systems increasingly use discretized self-supervised speech representations for compression and integration with token-based models, yet their impact on emotional information remains unclear. We study how residual vector quantization (RVQ) reshapes emotional information in discrete speech representations from both representation- and task-level perspectives. Our analysis shows that aggressive compression disproportionately degrades emotion, with uneven loss across emotion classes and model architectures. To address this, we introduce emotion-aware quantization using emotion-specific and emotion-biased codebooks, improving the preservation of both hard and soft emotion perception. We further propose Emo-Q, a lightweight routed quantization method that selects emotion-specialized codebooks, improving emotion recognition performance at lower bitrates. These results highlight the importance of emotion-aware discretization for robust affective speech processing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_21224 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Emotion-Aware Quantization for Discrete Speech Representations: An Analysis of Emotion Preservation Zhou, Haoguang Wang, Siyi Wu, Jingyao Bailey, James Dang, Ting Sound Modern speech systems increasingly use discretized self-supervised speech representations for compression and integration with token-based models, yet their impact on emotional information remains unclear. We study how residual vector quantization (RVQ) reshapes emotional information in discrete speech representations from both representation- and task-level perspectives. Our analysis shows that aggressive compression disproportionately degrades emotion, with uneven loss across emotion classes and model architectures. To address this, we introduce emotion-aware quantization using emotion-specific and emotion-biased codebooks, improving the preservation of both hard and soft emotion perception. We further propose Emo-Q, a lightweight routed quantization method that selects emotion-specialized codebooks, improving emotion recognition performance at lower bitrates. These results highlight the importance of emotion-aware discretization for robust affective speech processing. |
| title | Emotion-Aware Quantization for Discrete Speech Representations: An Analysis of Emotion Preservation |
| topic | Sound |
| url | https://arxiv.org/abs/2603.21224 |