LI-TTA: Language Informed Test-Time Adaptation for Automatic Speech Recognition
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914909407674368 |
|---|---|
| author | Yoon, Eunseop Yoon, Hee Suk Harvill, John Hasegawa-Johnson, Mark Yoo, Chang D. |
| author_facet | Yoon, Eunseop Yoon, Hee Suk Harvill, John Hasegawa-Johnson, Mark Yoo, Chang D. |
| contents | Test-Time Adaptation (TTA) has emerged as a crucial solution to the domain shift challenge, wherein the target environment diverges from the original training environment. A prime exemplification is TTA for Automatic Speech Recognition (ASR), which enhances model performance by leveraging output prediction entropy minimization as a self-supervision signal. However, a key limitation of this self-supervision lies in its primary focus on acoustic features, with minimal attention to the linguistic properties of the input. To address this gap, we propose Language Informed Test-Time Adaptation (LI-TTA), which incorporates linguistic insights during TTA for ASR. LI-TTA integrates corrections from an external language model to merge linguistic with acoustic information by minimizing the CTC loss from the correction alongside the standard TTA loss. With extensive experiments, we show that LI-TTA effectively improves the performance of TTA for ASR in various distribution shift situations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_05769 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LI-TTA: Language Informed Test-Time Adaptation for Automatic Speech Recognition Yoon, Eunseop Yoon, Hee Suk Harvill, John Hasegawa-Johnson, Mark Yoo, Chang D. Computation and Language Sound Audio and Speech Processing Test-Time Adaptation (TTA) has emerged as a crucial solution to the domain shift challenge, wherein the target environment diverges from the original training environment. A prime exemplification is TTA for Automatic Speech Recognition (ASR), which enhances model performance by leveraging output prediction entropy minimization as a self-supervision signal. However, a key limitation of this self-supervision lies in its primary focus on acoustic features, with minimal attention to the linguistic properties of the input. To address this gap, we propose Language Informed Test-Time Adaptation (LI-TTA), which incorporates linguistic insights during TTA for ASR. LI-TTA integrates corrections from an external language model to merge linguistic with acoustic information by minimizing the CTC loss from the correction alongside the standard TTA loss. With extensive experiments, we show that LI-TTA effectively improves the performance of TTA for ASR in various distribution shift situations. |
| title | LI-TTA: Language Informed Test-Time Adaptation for Automatic Speech Recognition |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2408.05769 |