Ara-Best-RQ: Multi Dialectal Arabic SSL
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910065713217536 |
|---|---|
| author | Elleuch, Haroun Whetten, Ryan Mdhaffar, Salima Estève, Yannick Bougares, Fethi |
| author_facet | Elleuch, Haroun Whetten, Ryan Mdhaffar, Salima Estève, Yannick Bougares, Fethi |
| contents | We present Ara-BEST-RQ, a family of self-supervised learning (SSL) models specifically designed for multi-dialectal Arabic speech processing. Leveraging 5,640 hours of crawled Creative Commons speech and combining it with publicly available datasets, we pre-train conformer-based BEST-RQ models up to 600M parameters. Our models are evaluated on dialect identification (DID) and automatic speech recognition (ASR) tasks, achieving state-of-the-art performance on the former while using fewer parameters than competing models. We demonstrate that family-targeted pre-training on Arabic dialects significantly improves downstream performance compared to multilingual or monolingual models trained on non-Arabic data. All models, code, and pre-processed datasets will be publicly released to support reproducibility and further research in Arabic speech technologies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_21900 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Ara-Best-RQ: Multi Dialectal Arabic SSL Elleuch, Haroun Whetten, Ryan Mdhaffar, Salima Estève, Yannick Bougares, Fethi Computation and Language We present Ara-BEST-RQ, a family of self-supervised learning (SSL) models specifically designed for multi-dialectal Arabic speech processing. Leveraging 5,640 hours of crawled Creative Commons speech and combining it with publicly available datasets, we pre-train conformer-based BEST-RQ models up to 600M parameters. Our models are evaluated on dialect identification (DID) and automatic speech recognition (ASR) tasks, achieving state-of-the-art performance on the former while using fewer parameters than competing models. We demonstrate that family-targeted pre-training on Arabic dialects significantly improves downstream performance compared to multilingual or monolingual models trained on non-Arabic data. All models, code, and pre-processed datasets will be publicly released to support reproducibility and further research in Arabic speech technologies. |
| title | Ara-Best-RQ: Multi Dialectal Arabic SSL |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.21900 |