Exploring Energy-Based Models for Out-of-Distribution Detection in Dialect Identification
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913405504323584 |
|---|---|
| author | Hao, Yaqian Hu, Chenguang Gao, Yingying Zhang, Shilei Feng, Junlan |
| author_facet | Hao, Yaqian Hu, Chenguang Gao, Yingying Zhang, Shilei Feng, Junlan |
| contents | The diverse nature of dialects presents challenges for models trained on specific linguistic patterns, rendering them susceptible to errors when confronted with unseen or out-of-distribution (OOD) data. This study introduces a novel margin-enhanced joint energy model (MEJEM) tailored specifically for OOD detection in dialects. By integrating a generative model and the energy margin loss, our approach aims to enhance the robustness of dialect identification systems. Furthermore, we explore two OOD scores for OOD dialect detection, and our findings conclusively demonstrate that the energy score outperforms the softmax score. Leveraging Sharpness-Aware Minimization to optimize the training process of the joint model, we enhance model generalization by minimizing both loss and sharpness. Experiments conducted on dialect identification tasks validate the efficacy of Energy-Based Models and provide valuable insights into their performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_18067 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploring Energy-Based Models for Out-of-Distribution Detection in Dialect Identification Hao, Yaqian Hu, Chenguang Gao, Yingying Zhang, Shilei Feng, Junlan Computation and Language Audio and Speech Processing The diverse nature of dialects presents challenges for models trained on specific linguistic patterns, rendering them susceptible to errors when confronted with unseen or out-of-distribution (OOD) data. This study introduces a novel margin-enhanced joint energy model (MEJEM) tailored specifically for OOD detection in dialects. By integrating a generative model and the energy margin loss, our approach aims to enhance the robustness of dialect identification systems. Furthermore, we explore two OOD scores for OOD dialect detection, and our findings conclusively demonstrate that the energy score outperforms the softmax score. Leveraging Sharpness-Aware Minimization to optimize the training process of the joint model, we enhance model generalization by minimizing both loss and sharpness. Experiments conducted on dialect identification tasks validate the efficacy of Energy-Based Models and provide valuable insights into their performance. |
| title | Exploring Energy-Based Models for Out-of-Distribution Detection in Dialect Identification |
| topic | Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2406.18067 |