Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.08399 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916892514451456 |
|---|---|
| author | Aihara, Ryo Masuyama, Yoshiki Wichern, Gordon Germain, François G. Roux, Jonathan Le |
| author_facet | Aihara, Ryo Masuyama, Yoshiki Wichern, Gordon Germain, François G. Roux, Jonathan Le |
| contents | Neural audio codecs (NACs), which use neural networks to generate compact audio representations, have garnered interest for their applicability to many downstream tasks -- especially quantized codecs due to their compatibility with large language models. However, unlike text, speech conveys not only linguistic content but also rich paralinguistic features. Encoding these elements in an entangled fashion may be suboptimal, as it limits flexibility. For instance, voice conversion (VC) aims to convert speaker characteristics while preserving the original linguistic content, which requires a disentangled representation. Inspired by VC methods utilizing $k$-means quantization with self-supervised features to disentangle phonetic information, we develop a discrete NAC capable of structured disentanglement. Experimental evaluations show that our approach achieves reconstruction performance on par with conventional NACs that do not explicitly perform disentanglement, while also matching the effectiveness of conventional VC techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_08399 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploring Disentangled Neural Speech Codecs from Self-Supervised Representations Aihara, Ryo Masuyama, Yoshiki Wichern, Gordon Germain, François G. Roux, Jonathan Le Audio and Speech Processing Signal Processing Neural audio codecs (NACs), which use neural networks to generate compact audio representations, have garnered interest for their applicability to many downstream tasks -- especially quantized codecs due to their compatibility with large language models. However, unlike text, speech conveys not only linguistic content but also rich paralinguistic features. Encoding these elements in an entangled fashion may be suboptimal, as it limits flexibility. For instance, voice conversion (VC) aims to convert speaker characteristics while preserving the original linguistic content, which requires a disentangled representation. Inspired by VC methods utilizing $k$-means quantization with self-supervised features to disentangle phonetic information, we develop a discrete NAC capable of structured disentanglement. Experimental evaluations show that our approach achieves reconstruction performance on par with conventional NACs that do not explicitly perform disentanglement, while also matching the effectiveness of conventional VC techniques. |
| title | Exploring Disentangled Neural Speech Codecs from Self-Supervised Representations |
| topic | Audio and Speech Processing Signal Processing |
| url | https://arxiv.org/abs/2508.08399 |