KoALa-Bench: Evaluating Large Audio Language Models on Korean Speech Understanding and Faithfulness
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915948333629440 |
|---|---|
| author | Kim, Jinyoung Lim, Hyeongsoo Seo, Eunseo Jang, Minho Choi, Keunwoo Shin, Seungyoun Yoon, Ji Won |
| author_facet | Kim, Jinyoung Lim, Hyeongsoo Seo, Eunseo Jang, Minho Choi, Keunwoo Shin, Seungyoun Yoon, Ji Won |
| contents | Recent advances in large audio language models (LALMs) have enabled multilingual speech understanding. However, benchmarks for evaluating LALMs remain scarce for non-English languages, with Korean being one such underexplored case. In this paper, we introduce KoALa-Bench, a comprehensive benchmark for evaluating Korean speech understanding and speech faithfulness of LALMs. In particular, KoALa-Bench comprises six tasks. Four tasks evaluate fundamental speech understanding capabilities, including automatic speech recognition, speech translation, speech question answering, and speech instruction following, while the remaining two tasks evaluate speech faithfulness, motivated by our observation that several LALMs often fail to fully leverage the speech modality. Furthermore, to reflect Korea-specific knowledge, our benchmark incorporates listening questions from the Korean college scholastic ability test as well as content covering Korean cultural domains. We conduct extensive experiments across six models, including both white-box and black-box ones. Our benchmark, evaluation code, and leaderboard are publicly available at https://ksbench.github.io/Korean-Benchmark/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_19782 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | KoALa-Bench: Evaluating Large Audio Language Models on Korean Speech Understanding and Faithfulness Kim, Jinyoung Lim, Hyeongsoo Seo, Eunseo Jang, Minho Choi, Keunwoo Shin, Seungyoun Yoon, Ji Won Computation and Language Artificial Intelligence Sound Audio and Speech Processing Recent advances in large audio language models (LALMs) have enabled multilingual speech understanding. However, benchmarks for evaluating LALMs remain scarce for non-English languages, with Korean being one such underexplored case. In this paper, we introduce KoALa-Bench, a comprehensive benchmark for evaluating Korean speech understanding and speech faithfulness of LALMs. In particular, KoALa-Bench comprises six tasks. Four tasks evaluate fundamental speech understanding capabilities, including automatic speech recognition, speech translation, speech question answering, and speech instruction following, while the remaining two tasks evaluate speech faithfulness, motivated by our observation that several LALMs often fail to fully leverage the speech modality. Furthermore, to reflect Korea-specific knowledge, our benchmark incorporates listening questions from the Korean college scholastic ability test as well as content covering Korean cultural domains. We conduct extensive experiments across six models, including both white-box and black-box ones. Our benchmark, evaluation code, and leaderboard are publicly available at https://ksbench.github.io/Korean-Benchmark/. |
| title | KoALa-Bench: Evaluating Large Audio Language Models on Korean Speech Understanding and Faithfulness |
| topic | Computation and Language Artificial Intelligence Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2604.19782 |