AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914555393736704 |
|---|---|
| author | Shi, Jiacheng Du, Hongfei Song, Xinyuan Hong, Y. Alicia Zhang, Yanfu Gao, Ye |
| author_facet | Shi, Jiacheng Du, Hongfei Song, Xinyuan Hong, Y. Alicia Zhang, Yanfu Gao, Ye |
| contents | Neural speech codecs provide discrete representations for speech language models, but emotional cues are often degraded during quantization. Existing codecs mainly optimize acoustic reconstruction, leaving emotion expressiveness insufficiently modeled at the representation level. We propose an emotion-guided neural speech codec that explicitly preserves emotional information while maintaining semantic fidelity and prosodic naturalness. Our framework combines emotion-semantic guided latent modulation, relation-preserving emotional-semantic distillation, and emotion-weighted semantic alignment to retain emotionally salient cues under compression. Extensive evaluations across speech reconstruction, emotion recognition, and downstream text-to-speech generation demonstrate improved emotion consistency and perceptual quality without sacrificing content accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_11098 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling Shi, Jiacheng Du, Hongfei Song, Xinyuan Hong, Y. Alicia Zhang, Yanfu Gao, Ye Sound Neural speech codecs provide discrete representations for speech language models, but emotional cues are often degraded during quantization. Existing codecs mainly optimize acoustic reconstruction, leaving emotion expressiveness insufficiently modeled at the representation level. We propose an emotion-guided neural speech codec that explicitly preserves emotional information while maintaining semantic fidelity and prosodic naturalness. Our framework combines emotion-semantic guided latent modulation, relation-preserving emotional-semantic distillation, and emotion-weighted semantic alignment to retain emotionally salient cues under compression. Extensive evaluations across speech reconstruction, emotion recognition, and downstream text-to-speech generation demonstrate improved emotion consistency and perceptual quality without sacrificing content accuracy. |
| title | AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling |
| topic | Sound |
| url | https://arxiv.org/abs/2605.11098 |