Reducing the Offline-Streaming Gap for Unified ASR Transducer with Consistency Regularization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917425143873536 |
|---|---|
| author | Andrusenko, Andrei Bataev, Vladimir Grigoryan, Lilit Tadevosyan, Nune Lavrukhin, Vitaly Ginsburg, Boris |
| author_facet | Andrusenko, Andrei Bataev, Vladimir Grigoryan, Lilit Tadevosyan, Nune Lavrukhin, Vitaly Ginsburg, Boris |
| contents | Unification of automatic speech recognition (ASR) systems reduces development and maintenance costs, but training a single model to perform well in both offline and low-latency streaming settings remains challenging. We present a Unified ASR framework for Transducer (RNNT) training that supports both offline and streaming decoding within a single model, using chunk-limited attention with right context and dynamic chunked convolutions. To further close the gap between offline and streaming performance, we introduce an efficient Triton implementation of mode-consistency regularization for RNNT (MCR-RNNT), which encourages agreement across training modes. Experiments show that the proposed approach improves streaming accuracy at low latency while preserving offline performance and scaling to larger model sizes and training datasets. The proposed Unified ASR framework and the English model checkpoint are open-sourced. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_19079 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Reducing the Offline-Streaming Gap for Unified ASR Transducer with Consistency Regularization Andrusenko, Andrei Bataev, Vladimir Grigoryan, Lilit Tadevosyan, Nune Lavrukhin, Vitaly Ginsburg, Boris Audio and Speech Processing Artificial Intelligence Computation and Language Human-Computer Interaction Unification of automatic speech recognition (ASR) systems reduces development and maintenance costs, but training a single model to perform well in both offline and low-latency streaming settings remains challenging. We present a Unified ASR framework for Transducer (RNNT) training that supports both offline and streaming decoding within a single model, using chunk-limited attention with right context and dynamic chunked convolutions. To further close the gap between offline and streaming performance, we introduce an efficient Triton implementation of mode-consistency regularization for RNNT (MCR-RNNT), which encourages agreement across training modes. Experiments show that the proposed approach improves streaming accuracy at low latency while preserving offline performance and scaling to larger model sizes and training datasets. The proposed Unified ASR framework and the English model checkpoint are open-sourced. |
| title | Reducing the Offline-Streaming Gap for Unified ASR Transducer with Consistency Regularization |
| topic | Audio and Speech Processing Artificial Intelligence Computation and Language Human-Computer Interaction |
| url | https://arxiv.org/abs/2604.19079 |