Universal Speech Content Factorization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917327865380864 |
|---|---|
| author | Xinyuan, Henry Li Cai, Zexin Zhang, Lin García-Perera, Leibny Paola Sisman, Berrak Khudanpur, Sanjeev Andrews, Nicholas Wiesner, Matthew |
| author_facet | Xinyuan, Henry Li Cai, Zexin Zhang, Lin García-Perera, Leibny Paola Sisman, Berrak Khudanpur, Sanjeev Andrews, Nicholas Wiesner, Matthew |
| contents | We propose Universal Speech Content Factorization (USCF), a simple and invertible linear method for extracting a low-rank speech representation in which speaker timbre is suppressed while phonetic content is preserved. USCF extends Speech Content Factorization, a closed-set voice conversion (VC) method, to an open-set setting by learning a universal speech-to-content mapping via least-squares optimization and deriving speaker-specific transformations from only a few seconds of target speech. We show through embedding analysis that USCF effectively removes speaker-dependent variation. As a zero-shot VC system, USCF achieves competitive intelligibility, naturalness, and speaker similarity compared to methods that require substantially more target-speaker data or additional neural training. Finally, we demonstrate that as a training-efficient timbre-disentangled speech feature, USCF features can serve as the acoustic representation for training timbre-prompted text-to-speech models. Speech samples and code are publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_08977 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Universal Speech Content Factorization Xinyuan, Henry Li Cai, Zexin Zhang, Lin García-Perera, Leibny Paola Sisman, Berrak Khudanpur, Sanjeev Andrews, Nicholas Wiesner, Matthew Audio and Speech Processing Sound We propose Universal Speech Content Factorization (USCF), a simple and invertible linear method for extracting a low-rank speech representation in which speaker timbre is suppressed while phonetic content is preserved. USCF extends Speech Content Factorization, a closed-set voice conversion (VC) method, to an open-set setting by learning a universal speech-to-content mapping via least-squares optimization and deriving speaker-specific transformations from only a few seconds of target speech. We show through embedding analysis that USCF effectively removes speaker-dependent variation. As a zero-shot VC system, USCF achieves competitive intelligibility, naturalness, and speaker similarity compared to methods that require substantially more target-speaker data or additional neural training. Finally, we demonstrate that as a training-efficient timbre-disentangled speech feature, USCF features can serve as the acoustic representation for training timbre-prompted text-to-speech models. Speech samples and code are publicly available. |
| title | Universal Speech Content Factorization |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2603.08977 |