What do self-supervised speech models know about Dutch? Analyzing advantages of language-specific pre-training
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913935545860096 |
|---|---|
| author | Kloots, Marianne de Heer Mohebbi, Hosein Pouw, Charlotte Shen, Gaofei Zuidema, Willem Bentum, Martijn |
| author_facet | Kloots, Marianne de Heer Mohebbi, Hosein Pouw, Charlotte Shen, Gaofei Zuidema, Willem Bentum, Martijn |
| contents | How language-specific are speech representations learned by self-supervised models? Existing work has shown that a range of linguistic features can be successfully decoded from end-to-end models trained only on speech recordings. However, it's less clear to what extent pre-training on specific languages improves language-specific linguistic information. Here we test the encoding of Dutch phonetic and lexical information in internal representations of self-supervised Wav2Vec2 models. Pre-training exclusively on Dutch improves the representation of Dutch linguistic features as compared to pre-training on similar amounts of English or larger amounts of multilingual data. This language-specific advantage is well-detected by trained clustering or classification probes, and partially observable using zero-shot metrics. Furthermore, the language-specific benefit on linguistic feature encoding aligns with downstream performance on Automatic Speech Recognition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_00981 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | What do self-supervised speech models know about Dutch? Analyzing advantages of language-specific pre-training Kloots, Marianne de Heer Mohebbi, Hosein Pouw, Charlotte Shen, Gaofei Zuidema, Willem Bentum, Martijn Computation and Language Artificial Intelligence Sound Audio and Speech Processing How language-specific are speech representations learned by self-supervised models? Existing work has shown that a range of linguistic features can be successfully decoded from end-to-end models trained only on speech recordings. However, it's less clear to what extent pre-training on specific languages improves language-specific linguistic information. Here we test the encoding of Dutch phonetic and lexical information in internal representations of self-supervised Wav2Vec2 models. Pre-training exclusively on Dutch improves the representation of Dutch linguistic features as compared to pre-training on similar amounts of English or larger amounts of multilingual data. This language-specific advantage is well-detected by trained clustering or classification probes, and partially observable using zero-shot metrics. Furthermore, the language-specific benefit on linguistic feature encoding aligns with downstream performance on Automatic Speech Recognition. |
| title | What do self-supervised speech models know about Dutch? Analyzing advantages of language-specific pre-training |
| topic | Computation and Language Artificial Intelligence Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2506.00981 |