Benchmarking Prosody Encoding in Discrete Speech Tokens
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913993224880128 |
|---|---|
| author | Onda, Kentaro Fukayama, Satoru Saito, Daisuke Minematsu, Nobuaki |
| author_facet | Onda, Kentaro Fukayama, Satoru Saito, Daisuke Minematsu, Nobuaki |
| contents | Recently, discrete tokens derived from self-supervised learning (SSL) models via k-means clustering have been actively studied as pseudo-text in speech language models and as efficient intermediate representations for various tasks. However, these discrete tokens are typically learned in advance, separately from the training of language models or downstream tasks. As a result, choices related to discretization, such as the SSL model used or the number of clusters, must be made heuristically. In particular, speech language models are expected to understand and generate responses that reflect not only the semantic content but also prosodic features. Yet, there has been limited research on the ability of discrete tokens to capture prosodic information. To address this gap, this study conducts a comprehensive analysis focusing on prosodic encoding based on their sensitivity to the artificially modified prosody, aiming to provide practical guidelines for designing discrete tokens. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_11224 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Benchmarking Prosody Encoding in Discrete Speech Tokens Onda, Kentaro Fukayama, Satoru Saito, Daisuke Minematsu, Nobuaki Sound Computation and Language Audio and Speech Processing Recently, discrete tokens derived from self-supervised learning (SSL) models via k-means clustering have been actively studied as pseudo-text in speech language models and as efficient intermediate representations for various tasks. However, these discrete tokens are typically learned in advance, separately from the training of language models or downstream tasks. As a result, choices related to discretization, such as the SSL model used or the number of clusters, must be made heuristically. In particular, speech language models are expected to understand and generate responses that reflect not only the semantic content but also prosodic features. Yet, there has been limited research on the ability of discrete tokens to capture prosodic information. To address this gap, this study conducts a comprehensive analysis focusing on prosodic encoding based on their sensitivity to the artificially modified prosody, aiming to provide practical guidelines for designing discrete tokens. |
| title | Benchmarking Prosody Encoding in Discrete Speech Tokens |
| topic | Sound Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2508.11224 |