A Unified Neural Codec Language Model for Selective Editable Text to Speech Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909994285268992 |
|---|---|
| author | Pei, Hanchen Liu, Shujie Liu, Yanqing Yu, Jianwei Qian, Yuanhang Huang, Gongping Zhao, Sheng Lu, Yan |
| author_facet | Pei, Hanchen Liu, Shujie Liu, Yanqing Yu, Jianwei Qian, Yuanhang Huang, Gongping Zhao, Sheng Lu, Yan |
| contents | Neural codec language models achieve impressive zero-shot Text-to-Speech (TTS) by fully imitating the acoustic characteristics of a short speech prompt, including timbre, prosody, and paralinguistic information. However, such holistic imitation limits their ability to isolate and control individual attributes. In this paper, we present a unified codec language model SpeechEdit that extends zero-shot TTS with a selective control mechanism. By default, SpeechEdit reproduces the complete acoustic profile inferred from the speech prompt, but it selectively overrides only the attributes specified by explicit control instructions. To enable controllable modeling, SpeechEdit is trained on our newly constructed LibriEdit dataset, which provides delta (difference-aware) training pairs derived from LibriHeavy. Experimental results show that our approach maintains naturalness and robustness while offering flexible and localized control over desired attributes. Audio samples are available at https://speech-editing.github.io/speech-editing/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_12480 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Unified Neural Codec Language Model for Selective Editable Text to Speech Generation Pei, Hanchen Liu, Shujie Liu, Yanqing Yu, Jianwei Qian, Yuanhang Huang, Gongping Zhao, Sheng Lu, Yan Sound Audio and Speech Processing Neural codec language models achieve impressive zero-shot Text-to-Speech (TTS) by fully imitating the acoustic characteristics of a short speech prompt, including timbre, prosody, and paralinguistic information. However, such holistic imitation limits their ability to isolate and control individual attributes. In this paper, we present a unified codec language model SpeechEdit that extends zero-shot TTS with a selective control mechanism. By default, SpeechEdit reproduces the complete acoustic profile inferred from the speech prompt, but it selectively overrides only the attributes specified by explicit control instructions. To enable controllable modeling, SpeechEdit is trained on our newly constructed LibriEdit dataset, which provides delta (difference-aware) training pairs derived from LibriHeavy. Experimental results show that our approach maintains naturalness and robustness while offering flexible and localized control over desired attributes. Audio samples are available at https://speech-editing.github.io/speech-editing/. |
| title | A Unified Neural Codec Language Model for Selective Editable Text to Speech Generation |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2601.12480 |