Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.05787 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915097144721408 |
|---|---|
| author | Baas, Matthew Scholtz, Pieter Mehta, Arnav Dyson, Elliott Prakash, Akshat Kamper, Herman |
| author_facet | Baas, Matthew Scholtz, Pieter Mehta, Arnav Dyson, Elliott Prakash, Akshat Kamper, Herman |
| contents | Codec-based text-to-speech (TTS) models have shown impressive quality with zero-shot voice cloning abilities. However, they often struggle with more expressive references or complex text inputs. We present MARS6, a robust encoder-decoder transformer for rapid, expressive TTS. MARS6 is built on recent improvements in spoken language modelling. Utilizing a hierarchical setup for its decoder, new speech tokens are processed at a rate of only 12 Hz, enabling efficient modelling of long-form text while retaining reconstruction quality. We combine several recent training and inference techniques to reduce repetitive generation and improve output stability and quality. This enables the 70M-parameter MARS6 to achieve similar performance to models many times larger. We show this in objective and subjective evaluations, comparing TTS output quality and reference speaker cloning ability. Project page: https://camb-ai.github.io/mars6-turbo/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_05787 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MARS6: A Small and Robust Hierarchical-Codec Text-to-Speech Model Baas, Matthew Scholtz, Pieter Mehta, Arnav Dyson, Elliott Prakash, Akshat Kamper, Herman Audio and Speech Processing Computation and Language Codec-based text-to-speech (TTS) models have shown impressive quality with zero-shot voice cloning abilities. However, they often struggle with more expressive references or complex text inputs. We present MARS6, a robust encoder-decoder transformer for rapid, expressive TTS. MARS6 is built on recent improvements in spoken language modelling. Utilizing a hierarchical setup for its decoder, new speech tokens are processed at a rate of only 12 Hz, enabling efficient modelling of long-form text while retaining reconstruction quality. We combine several recent training and inference techniques to reduce repetitive generation and improve output stability and quality. This enables the 70M-parameter MARS6 to achieve similar performance to models many times larger. We show this in objective and subjective evaluations, comparing TTS output quality and reference speaker cloning ability. Project page: https://camb-ai.github.io/mars6-turbo/ |
| title | MARS6: A Small and Robust Hierarchical-Codec Text-to-Speech Model |
| topic | Audio and Speech Processing Computation and Language |
| url | https://arxiv.org/abs/2501.05787 |