Cover Image

KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kim, Haechan, Chung, Seungjun, Park, Inkyu, Lee, Jihoo, Lee, Jonghyun
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Raon-Speech Technical Report
by: Kim, Beomsoo, et al.
Published: (2026)

KoALa-Bench: Evaluating Large Audio Language Models on Korean Speech Understanding and Faithfulness
by: Kim, Jinyoung, et al.
Published: (2026)

ESPnet-SpeechLM: An Open Speech Language Model Toolkit
by: Tian, Jinchuan, et al.
Published: (2025)

HAE-RAE Bench: Evaluation of Korean Knowledge in Language Models
by: Son, Guijin, et al.
Published: (2023)

Open Ko-LLM Leaderboard2: Bridging Foundational and Practical Evaluation for Korean LLMs
by: Kim, Hyeonwoo, et al.
Published: (2024)

Evaluating Large language models on Understanding Korean indirect Speech acts
by: Koo, Youngeun, et al.
Published: (2025)

KoDialogBench: Evaluating Conversational Understanding of Language Models with Korean Dialogue Benchmark
by: Jang, Seongbo, et al.
Published: (2024)

Nunchi-Bench: Benchmarking Language Models on Cultural Reasoning with a Focus on Korean Superstition
by: Kim, Kyuhee, et al.
Published: (2025)

WildSpeech-Bench: Benchmarking End-to-End SpeechLLMs in the Wild
by: Zhang, Linhao, et al.
Published: (2025)

MathSpeech: Leveraging Small LMs for Accurate Conversion in Mathematical Speech-to-Formula
by: Hyeon, Sieun, et al.
Published: (2024)

Speech Robust Bench: A Robustness Benchmark For Speech Recognition
by: Shah, Muhammad A., et al.
Published: (2024)

ISD-Agent-Bench: A Comprehensive Benchmark for Evaluating LLM-based Instructional Design Agents
by: Jeon, YoungHoon, et al.
Published: (2026)

OLKAVS: An Open Large-Scale Korean Audio-Visual Speech Dataset
by: Park, Jeongkyun, et al.
Published: (2023)

VocalBench-DF: A Benchmark for Evaluating Speech LLM Robustness to Disfluency
by: Liu, Hongcheng, et al.
Published: (2025)

Lightweight Audio Segmentation for Long-form Speech Translation
by: Lee, Jaesong, et al.
Published: (2024)

Speech Corpus for Korean Children with Autism Spectrum Disorder: Towards Automatic Assessment Systems
by: Lee, Seonwoo, et al.
Published: (2024)

SpeechParaling-Bench: A Comprehensive Benchmark for Paralinguistic-Aware Speech Generation
by: Liu, Ruohan, et al.
Published: (2026)

BlasBench: An Open Benchmark for Irish Speech Recognition
by: Raj, Jyoutir, et al.
Published: (2026)

LearnerVoice: A Dataset of Non-Native English Learners' Spontaneous Speech
by: Kim, Haechan, et al.
Published: (2024)

VocalBench: Benchmarking the Vocal Conversational Abilities for Speech Interaction Models
by: Liu, Heyang, et al.
Published: (2025)

ParliaBench: An Evaluation and Benchmarking Framework for LLM-Generated Parliamentary Speech
by: Koniaris, Marios, et al.
Published: (2025)

DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors
by: Lee, Keon, et al.
Published: (2024)

VCB Bench: An Evaluation Benchmark for Audio-Grounded Large Language Model Conversational Agents
by: Hu, Jiliang, et al.
Published: (2025)

KULTURE Bench: A Benchmark for Assessing Language Model in Korean Cultural Context
by: Wang, Xiaonan, et al.
Published: (2024)

SyncVSR: Data-Efficient Visual Speech Recognition with End-to-End Crossmodal Audio Token Synchronization
by: Ahn, Young Jin, et al.
Published: (2024)

ScholarBench: A Bilingual Benchmark for Abstraction, Comprehension, and Reasoning Evaluation in Academic Contexts
by: Noh, Dongwon, et al.
Published: (2025)

CS3-Bench: Evaluating and Enhancing Speech-to-Speech LLMs for Mandarin-English Code-Switching
by: Liu, Heyang, et al.
Published: (2025)

HiKE: Hierarchical Evaluation Framework for Korean-English Code-Switching Speech Recognition
by: Paik, Gio, et al.
Published: (2025)

FunctionChat-Bench: Comprehensive Evaluation of Language Models' Generative Capabilities in Korean Tool-use Dialogs
by: Lee, Shinbok, et al.
Published: (2024)

VocalBench-zh: Decomposing and Benchmarking the Speech Conversational Abilities in Mandarin Context
by: Liu, Heyang, et al.
Published: (2025)

ContextASR-Bench: A Massive Contextual Speech Recognition Benchmark
by: Wang, He, et al.
Published: (2025)

Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBench
by: Perlitz, Yotam, et al.
Published: (2024)

Evaluating Automatic Speech Recognition Systems for Korean Meteorological Experts
by: Park, ChaeHun, et al.
Published: (2024)

StreamBench: Towards Benchmarking Continuous Improvement of Language Agents
by: Wu, Cheng-Kuang, et al.
Published: (2024)

StyleBench: Evaluating Speech Language Models on Conversational Speaking Style Control
by: Zhao, Haishu, et al.
Published: (2026)

K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts
by: Lee, Nahyun, et al.
Published: (2026)

Reasoning Beyond Majority Vote: An Explainable SpeechLM Framework for Speech Emotion Recognition
by: Su, Bo-Hao, et al.
Published: (2025)

FHIR-AgentBench: Benchmarking LLM Agents for Realistic Interoperable EHR Question Answering
by: Lee, Gyubok, et al.
Published: (2025)

Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmark
by: Park, Chanjun, et al.
Published: (2024)

An Investigation Into Explainable Audio Hate Speech Detection
by: An, Jinmyeong, et al.
Published: (2024)