TASU2: Controllable CTC Simulation for Alignment and Low-Resource Adaptation of Speech LLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915927487938560 |
|---|---|
| author | Peng, Jing Wang, Chenghao Yang, Yi Qian, Lirong Li, Junjie Xi, Yu Wang, Shuai Yu, Kai |
| author_facet | Peng, Jing Wang, Chenghao Yang, Yi Qian, Lirong Li, Junjie Xi, Yu Wang, Shuai Yu, Kai |
| contents | Speech LLM post-training increasingly relies on efficient cross-modal alignment and robust low-resource adaptation, yet collecting large-scale audio-text pairs remains costly. Text-only alignment methods such as TASU reduce this burden by simulating CTC posteriors from transcripts, but they provide limited control over uncertainty and error rate, making curriculum design largely heuristic. We propose \textbf{TASU2}, a controllable CTC simulation framework that simulates CTC posterior distributions under a specified WER range, producing text-derived supervision that better matches the acoustic decoding interface. This enables principled post-training curricula that smoothly vary supervision difficulty without TTS. Across multiple source-to-target adaptation settings, TASU2 improves in-domain and out-of-domain recognition over TASU, and consistently outperforms strong baselines including text-only fine-tuning and TTS-based augmentation, while mitigating source-domain performance degradation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_08384 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TASU2: Controllable CTC Simulation for Alignment and Low-Resource Adaptation of Speech LLMs Peng, Jing Wang, Chenghao Yang, Yi Qian, Lirong Li, Junjie Xi, Yu Wang, Shuai Yu, Kai Audio and Speech Processing Artificial Intelligence Speech LLM post-training increasingly relies on efficient cross-modal alignment and robust low-resource adaptation, yet collecting large-scale audio-text pairs remains costly. Text-only alignment methods such as TASU reduce this burden by simulating CTC posteriors from transcripts, but they provide limited control over uncertainty and error rate, making curriculum design largely heuristic. We propose \textbf{TASU2}, a controllable CTC simulation framework that simulates CTC posterior distributions under a specified WER range, producing text-derived supervision that better matches the acoustic decoding interface. This enables principled post-training curricula that smoothly vary supervision difficulty without TTS. Across multiple source-to-target adaptation settings, TASU2 improves in-domain and out-of-domain recognition over TASU, and consistently outperforms strong baselines including text-only fine-tuning and TTS-based augmentation, while mitigating source-domain performance degradation. |
| title | TASU2: Controllable CTC Simulation for Alignment and Low-Resource Adaptation of Speech LLMs |
| topic | Audio and Speech Processing Artificial Intelligence |
| url | https://arxiv.org/abs/2604.08384 |