Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.00688 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911610558218240 |
|---|---|
| author | Zhu, Han Ye, Lingxuan Kang, Wei Yao, Zengwei Guo, Liyong Kuang, Fangjun Han, Zhifeng Zhuang, Weiji Lin, Long Povey, Daniel |
| author_facet | Zhu, Han Ye, Lingxuan Kang, Wei Yao, Zengwei Guo, Liyong Kuang, Fangjun Han, Zhifeng Zhuang, Weiji Lin, Long Povey, Daniel |
| contents | We present OmniVoice, a massively multilingual zero-shot text-to-speech (TTS) model that scales to over 600 languages. At its core is a novel diffusion language model-style discrete non-autoregressive (NAR) architecture. Unlike conventional discrete NAR models that suffer from performance bottlenecks in complex two-stage (text-to-semantic-to-acoustic) pipelines, OmniVoice directly maps text to multi-codebook acoustic tokens. This simplified approach is facilitated by two key technical innovations: (1) a full-codebook random masking strategy for efficient training, and (2) initialization from a pre-trained LLM to ensure superior intelligibility. By leveraging a 581k-hour multilingual dataset curated entirely from open-source data, OmniVoice achieves the broadest language coverage to date and delivers state-of-the-art performance across Chinese, English, and diverse multilingual benchmarks. Our code and pre-trained models are publicly available at https://github.com/k2-fsa/OmniVoice. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_00688 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models Zhu, Han Ye, Lingxuan Kang, Wei Yao, Zengwei Guo, Liyong Kuang, Fangjun Han, Zhifeng Zhuang, Weiji Lin, Long Povey, Daniel Computation and Language Audio and Speech Processing We present OmniVoice, a massively multilingual zero-shot text-to-speech (TTS) model that scales to over 600 languages. At its core is a novel diffusion language model-style discrete non-autoregressive (NAR) architecture. Unlike conventional discrete NAR models that suffer from performance bottlenecks in complex two-stage (text-to-semantic-to-acoustic) pipelines, OmniVoice directly maps text to multi-codebook acoustic tokens. This simplified approach is facilitated by two key technical innovations: (1) a full-codebook random masking strategy for efficient training, and (2) initialization from a pre-trained LLM to ensure superior intelligibility. By leveraging a 581k-hour multilingual dataset curated entirely from open-source data, OmniVoice achieves the broadest language coverage to date and delivers state-of-the-art performance across Chinese, English, and diverse multilingual benchmarks. Our code and pre-trained models are publicly available at https://github.com/k2-fsa/OmniVoice. |
| title | OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models |
| topic | Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2604.00688 |