Iterative refinement, not training objective, makes HuBERT behave differently from wav2vec 2.0
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918122172186624 |
|---|---|
| author | Huo, Robin Dunbar, Ewan |
| author_facet | Huo, Robin Dunbar, Ewan |
| contents | Self-supervised models for speech representation learning now see widespread use for their versatility and performance on downstream tasks, but the effect of model architecture on the linguistic information learned in their representations remains under-studied. This study investigates two such models, HuBERT and wav2vec 2.0, and minimally compares two of their architectural differences: training objective and iterative pseudo-label refinement through multiple training iterations. We find that differences in canonical correlation of hidden representations to word identity, phoneme identity, and speaker identity are explained by training iteration, not training objective. We suggest that future work investigate the reason for the effectiveness of iterative refinement in encoding linguistic information in self-supervised speech representations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_08110 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Iterative refinement, not training objective, makes HuBERT behave differently from wav2vec 2.0 Huo, Robin Dunbar, Ewan Computation and Language Sound Audio and Speech Processing Self-supervised models for speech representation learning now see widespread use for their versatility and performance on downstream tasks, but the effect of model architecture on the linguistic information learned in their representations remains under-studied. This study investigates two such models, HuBERT and wav2vec 2.0, and minimally compares two of their architectural differences: training objective and iterative pseudo-label refinement through multiple training iterations. We find that differences in canonical correlation of hidden representations to word identity, phoneme identity, and speaker identity are explained by training iteration, not training objective. We suggest that future work investigate the reason for the effectiveness of iterative refinement in encoding linguistic information in self-supervised speech representations. |
| title | Iterative refinement, not training objective, makes HuBERT behave differently from wav2vec 2.0 |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2508.08110 |