A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASR
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908886962798592 |
|---|---|
| author | Zheng, Yuang Chen, Dongxu Mei, Yuxiang Xu, Dongxing Chen, Jie Long, Yanhua |
| author_facet | Zheng, Yuang Chen, Dongxu Mei, Yuxiang Xu, Dongxing Chen, Jie Long, Yanhua |
| contents | Large-scale multilingual ASR (mASR) models such as Whisper achieve strong performance but incur high computational and latency costs, limiting their deployment on resource-constrained edge devices. In this study, we propose a lightweight and language-agnostic multilingual ASR system based on a CTC architecture with domain adaptation. Specifically, we introduce a Language-agnostic Hierarchical LoRA-MoE (HLoRA) framework integrated into an mHuBERT-CTC model, enabling end-to-end decoding via LID-posterior-driven LoRA routing. The hierarchical design consists of a multilingual shared LoRA for learning language-invariant acoustic representations and language-specific LoRA experts for modeling language-dependent characteristics. The proposed routing mechanism removes the need for prior language identity information or explicit language labels during inference, achieving true language-agnostic decoding. Experiments on MSR-86K and the MLC-SLM 2025 Challenge datasets demonstrate that HLoRA achieves comparable performance to two-stage inference approaches while reducing RTF by 11.7% and 8.2%, respectively, leading to improved decoding efficiency for low-resource mASR applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_00557 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASR Zheng, Yuang Chen, Dongxu Mei, Yuxiang Xu, Dongxing Chen, Jie Long, Yanhua Computation and Language Sound Audio and Speech Processing Large-scale multilingual ASR (mASR) models such as Whisper achieve strong performance but incur high computational and latency costs, limiting their deployment on resource-constrained edge devices. In this study, we propose a lightweight and language-agnostic multilingual ASR system based on a CTC architecture with domain adaptation. Specifically, we introduce a Language-agnostic Hierarchical LoRA-MoE (HLoRA) framework integrated into an mHuBERT-CTC model, enabling end-to-end decoding via LID-posterior-driven LoRA routing. The hierarchical design consists of a multilingual shared LoRA for learning language-invariant acoustic representations and language-specific LoRA experts for modeling language-dependent characteristics. The proposed routing mechanism removes the need for prior language identity information or explicit language labels during inference, achieving true language-agnostic decoding. Experiments on MSR-86K and the MLC-SLM 2025 Challenge datasets demonstrate that HLoRA achieves comparable performance to two-stage inference approaches while reducing RTF by 11.7% and 8.2%, respectively, leading to improved decoding efficiency for low-resource mASR applications. |
| title | A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASR |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2601.00557 |