Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.12101 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912587817418752 |
|---|---|
| author | Duret, Jarod Mdhaffar, Salima Laperrière, Gaëlle Whetten, Ryan Galametz, Audrey Kobus, Catherine Martin, Marion-Cécile Oleiwan, Jo Estève, Yannick |
| author_facet | Duret, Jarod Mdhaffar, Salima Laperrière, Gaëlle Whetten, Ryan Galametz, Audrey Kobus, Catherine Martin, Marion-Cécile Oleiwan, Jo Estève, Yannick |
| contents | In this study, we investigate the benefits of domain-specific self-supervised pre-training for both offline and streaming ASR in Air Traffic Control (ATC) environments. We train BEST-RQ models on 4.5k hours of unlabeled ATC data, then fine-tune on a smaller supervised ATC set. To enable real-time processing, we propose using chunked attention and dynamic convolutions, ensuring low-latency inference. We compare these in-domain SSL models against state-of-the-art, general-purpose speech encoders such as w2v-BERT 2.0 and HuBERT. Results show that domain-adapted pre-training substantially improves performance on standard ATC benchmarks, significantly reducing word error rates when compared to models trained on broad speech corpora. Furthermore, the proposed streaming approach further improves word error rate under tighter latency constraints, making it particularly suitable for safety-critical aviation applications. These findings highlight that specializing SSL representations for ATC data is a practical path toward more accurate and efficient ASR systems in real-world operational settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_12101 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | In-domain SSL pre-training and streaming ASR Duret, Jarod Mdhaffar, Salima Laperrière, Gaëlle Whetten, Ryan Galametz, Audrey Kobus, Catherine Martin, Marion-Cécile Oleiwan, Jo Estève, Yannick Computation and Language Artificial Intelligence In this study, we investigate the benefits of domain-specific self-supervised pre-training for both offline and streaming ASR in Air Traffic Control (ATC) environments. We train BEST-RQ models on 4.5k hours of unlabeled ATC data, then fine-tune on a smaller supervised ATC set. To enable real-time processing, we propose using chunked attention and dynamic convolutions, ensuring low-latency inference. We compare these in-domain SSL models against state-of-the-art, general-purpose speech encoders such as w2v-BERT 2.0 and HuBERT. Results show that domain-adapted pre-training substantially improves performance on standard ATC benchmarks, significantly reducing word error rates when compared to models trained on broad speech corpora. Furthermore, the proposed streaming approach further improves word error rate under tighter latency constraints, making it particularly suitable for safety-critical aviation applications. These findings highlight that specializing SSL representations for ATC data is a practical path toward more accurate and efficient ASR systems in real-world operational settings. |
| title | In-domain SSL pre-training and streaming ASR |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.12101 |