Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.03459 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916423479066624 |
|---|---|
| author | Zheng, Jiahao Ren, Jinke Xu, Peng Yuan, Zhihao Xu, Jie Wang, Fangxin Gui, Gui Cui, Shuguang |
| author_facet | Zheng, Jiahao Ren, Jinke Xu, Peng Yuan, Zhihao Xu, Jie Wang, Fangxin Gui, Gui Cui, Shuguang |
| contents | Semantic communication is a promising technology to improve communication efficiency by transmitting only the semantic information of the source data. However, traditional semantic communication methods primarily focus on data reconstruction tasks, which may not be efficient for emerging generative tasks such as text-to-speech (TTS) synthesis. To address this limitation, this paper develops a novel generative semantic communication framework for TTS synthesis, leveraging generative artificial intelligence technologies. Firstly, we utilize a pre-trained large speech model called WavLM and the residual vector quantization method to construct two semantic knowledge bases (KBs) at the transmitter and receiver, respectively. The KB at the transmitter enables effective semantic extraction, while the KB at the receiver facilitates lifelike speech synthesis. Then, we employ a transformer encoder and a diffusion model to achieve efficient semantic coding without introducing significant communication overhead. Finally, numerical results demonstrate that our framework achieves much higher fidelity for the generated speech than four baselines, in both cases with additive white Gaussian noise channel and Rayleigh fading channel. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03459 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Generative Semantic Communication for Text-to-Speech Synthesis Zheng, Jiahao Ren, Jinke Xu, Peng Yuan, Zhihao Xu, Jie Wang, Fangxin Gui, Gui Cui, Shuguang Sound Information Theory Machine Learning Audio and Speech Processing Semantic communication is a promising technology to improve communication efficiency by transmitting only the semantic information of the source data. However, traditional semantic communication methods primarily focus on data reconstruction tasks, which may not be efficient for emerging generative tasks such as text-to-speech (TTS) synthesis. To address this limitation, this paper develops a novel generative semantic communication framework for TTS synthesis, leveraging generative artificial intelligence technologies. Firstly, we utilize a pre-trained large speech model called WavLM and the residual vector quantization method to construct two semantic knowledge bases (KBs) at the transmitter and receiver, respectively. The KB at the transmitter enables effective semantic extraction, while the KB at the receiver facilitates lifelike speech synthesis. Then, we employ a transformer encoder and a diffusion model to achieve efficient semantic coding without introducing significant communication overhead. Finally, numerical results demonstrate that our framework achieves much higher fidelity for the generated speech than four baselines, in both cases with additive white Gaussian noise channel and Rayleigh fading channel. |
| title | Generative Semantic Communication for Text-to-Speech Synthesis |
| topic | Sound Information Theory Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2410.03459 |