voice2mode: Phonation Mode Classification in Singing using Self-Supervised Speech Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918339535699968 |
|---|---|
| author | Justus, Aju Ani Agrawal, Ruchit Kadiri, Sudarsana Reddy Narayanan, Shrikanth |
| author_facet | Justus, Aju Ani Agrawal, Ruchit Kadiri, Sudarsana Reddy Narayanan, Shrikanth |
| contents | We present voice2mode, a method for classification of four singing phonation modes (breathy, neutral (modal), flow, and pressed) using embeddings extracted from large self-supervised speech models. Prior work on singing phonation has relied on handcrafted signal features or task-specific neural nets; this work evaluates the transferability of speech foundation models to singing phonation classification. voice2mode extracts layer-wise representations from HuBERT and two wav2vec2 variants, applies global temporal pooling, and classifies the pooled embeddings with lightweight classifiers (SVM, XGBoost). Experiments on a publicly available soprano dataset (763 sustained vowel recordings, four labels) show that foundation-model features substantially outperform conventional spectral baselines (spectrogram, mel-spectrogram, MFCC). HuBERT embeddings obtained from early layers yield the best result (~95.7% accuracy with SVM), an absolute improvement of ~12-15% over the best traditional baseline. We also show layer-wise behaviour: lower layers, which retain acoustic/phonetic detail, are more effective than top layers specialized for Automatic Speech Recognition (ASR). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_13928 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | voice2mode: Phonation Mode Classification in Singing using Self-Supervised Speech Models Justus, Aju Ani Agrawal, Ruchit Kadiri, Sudarsana Reddy Narayanan, Shrikanth Sound Machine Learning We present voice2mode, a method for classification of four singing phonation modes (breathy, neutral (modal), flow, and pressed) using embeddings extracted from large self-supervised speech models. Prior work on singing phonation has relied on handcrafted signal features or task-specific neural nets; this work evaluates the transferability of speech foundation models to singing phonation classification. voice2mode extracts layer-wise representations from HuBERT and two wav2vec2 variants, applies global temporal pooling, and classifies the pooled embeddings with lightweight classifiers (SVM, XGBoost). Experiments on a publicly available soprano dataset (763 sustained vowel recordings, four labels) show that foundation-model features substantially outperform conventional spectral baselines (spectrogram, mel-spectrogram, MFCC). HuBERT embeddings obtained from early layers yield the best result (~95.7% accuracy with SVM), an absolute improvement of ~12-15% over the best traditional baseline. We also show layer-wise behaviour: lower layers, which retain acoustic/phonetic detail, are more effective than top layers specialized for Automatic Speech Recognition (ASR). |
| title | voice2mode: Phonation Mode Classification in Singing using Self-Supervised Speech Models |
| topic | Sound Machine Learning |
| url | https://arxiv.org/abs/2602.13928 |