Streaming Translation and Transcription Through Speech-to-Text Causal Alignment
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915856737370112 |
|---|---|
| author | Koshkin, Roman Haesung, Jeon Liu, Lianbo Shi, Hao Zhao, Mengjie Fujita, Yusuke Sudo, Yui |
| author_facet | Koshkin, Roman Haesung, Jeon Liu, Lianbo Shi, Hao Zhao, Mengjie Fujita, Yusuke Sudo, Yui |
| contents | Simultaneous machine translation (SiMT) has traditionally relied on offline machine translation models coupled with human-engineered heuristics or learned policies. We propose Hikari, a policy-free, fully end-to-end model that performs simultaneous speech-to-text translation and streaming transcription by encoding READ/WRITE decisions into a probabilistic WAIT token mechanism. We also introduce Decoder Time Dilation, a mechanism that reduces autoregressive overhead and ensures a balanced training distribution. Additionally, we present a supervised fine-tuning strategy that trains the model to recover from delays, significantly improving the quality-latency trade-off. Evaluated on English-to-Japanese, German, and Russian, Hikari achieves new state-of-the-art BLEU scores in both low- and high-latency regimes, outperforming recent baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_11578 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Streaming Translation and Transcription Through Speech-to-Text Causal Alignment Koshkin, Roman Haesung, Jeon Liu, Lianbo Shi, Hao Zhao, Mengjie Fujita, Yusuke Sudo, Yui Computation and Language Simultaneous machine translation (SiMT) has traditionally relied on offline machine translation models coupled with human-engineered heuristics or learned policies. We propose Hikari, a policy-free, fully end-to-end model that performs simultaneous speech-to-text translation and streaming transcription by encoding READ/WRITE decisions into a probabilistic WAIT token mechanism. We also introduce Decoder Time Dilation, a mechanism that reduces autoregressive overhead and ensures a balanced training distribution. Additionally, we present a supervised fine-tuning strategy that trains the model to recover from delays, significantly improving the quality-latency trade-off. Evaluated on English-to-Japanese, German, and Russian, Hikari achieves new state-of-the-art BLEU scores in both low- and high-latency regimes, outperforming recent baselines. |
| title | Streaming Translation and Transcription Through Speech-to-Text Causal Alignment |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.11578 |