Imagen de Portada

From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for LLM Evaluation

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Hong, Seokhee, Kim, Sunkyoung, Son, Guijin, Kim, Soyeon, Hong, Yeonjung, Lee, Jinsik
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

KMMLU: Measuring Massive Multitask Language Understanding in Korean
por: Son, Guijin, et al.
Publicado: (2024)

Cross-lingual QA: A Key to Unlocking In-context Cross-lingual Performance
por: Kim, Sunkyoung, et al.
Publicado: (2023)

BenchHub: A Unified Benchmark Suite for Holistic and Customizable LLM Evaluation
por: Kim, Eunsu, et al.
Publicado: (2025)

Redefining Evaluation Standards: A Unified Framework for Evaluating the Korean Capabilities of Language Models
por: Lee, Hanwool, et al.
Publicado: (2025)

Revisiting the UID Hypothesis in LLM Reasoning Traces
por: Gwak, Minju, et al.
Publicado: (2025)

Revisiting the Uniform Information Density Hypothesis in LLM Reasoning
por: Gwak, Minju, et al.
Publicado: (2025)

KFinEval-Pilot: A Comprehensive Benchmark Suite for Korean Financial Language Understanding
por: Hwang, Bokwang, et al.
Publicado: (2025)

Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmark
por: Park, Chanjun, et al.
Publicado: (2024)

BenchPreS: A Benchmark for Context-Aware Personalized Preference Selectivity of Persistent-Memory LLMs
por: Yoon, Sangyeon, et al.
Publicado: (2026)

EXAONE 3.0 7.8B Instruction Tuned Language Model
por: An, Soyoung, et al.
Publicado: (2024)

Open Ko-LLM Leaderboard2: Bridging Foundational and Practical Evaluation for Korean LLMs
por: Kim, Hyeonwoo, et al.
Publicado: (2024)

Reasoning Models Better Express Their Confidence
por: Yoon, Dongkeun, et al.
Publicado: (2025)

EXAONE Deep: Reasoning Enhanced Language Models
por: Bae, Kyunghoon, et al.
Publicado: (2025)

KoBBQ: Korean Bias Benchmark for Question Answering
por: Jin, Jiho, et al.
Publicado: (2023)

KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs
por: Kim, Haechan, et al.
Publicado: (2026)

KoBALT: Korean Benchmark For Advanced Linguistic Tasks
por: Shin, Hyopil, et al.
Publicado: (2025)

Nunchi-Bench: Benchmarking Language Models on Cultural Reasoning with a Focus on Korean Superstition
por: Kim, Kyuhee, et al.
Publicado: (2025)

KITE: A Benchmark for Evaluating Korean Instruction-Following Abilities in Large Language Models
por: Kim, Dongjun, et al.
Publicado: (2025)

On the Robustness of Reward Models for Language Model Alignment
por: Hong, Jiwoo, et al.
Publicado: (2025)

EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modes
por: Bae, Kyunghoon, et al.
Publicado: (2025)

Ko-PIQA: A Korean Physical Commonsense Reasoning Dataset with Cultural Context
por: Choi, Dasol, et al.
Publicado: (2025)

SLM-Based Agentic AI with P-C-G: Optimized for Korean Tool Use
por: Jeon, Changhyun, et al.
Publicado: (2025)

M-Prometheus: A Suite of Open Multilingual LLM Judges
por: Pombal, José, et al.
Publicado: (2025)

KOMBO: Korean Character Representations Based on the Combination Rules of Subcharacters
por: Kim, SungHo, et al.
Publicado: (2026)

KoACD: The First Korean Adolescent Dataset for Cognitive Distortion Analysis via Role-Switching Multi-LLM Negotiation
por: Kim, JunSeo, et al.
Publicado: (2025)

KatFishNet: Detecting LLM-Generated Korean Text through Linguistic Feature Analysis
por: Park, Shinwoo, et al.
Publicado: (2025)

KoCoSa: Korean Context-aware Sarcasm Detection Dataset
por: Kim, Yumin, et al.
Publicado: (2024)

KOFFVQA: An Objectively Evaluated Free-form VQA Benchmark for Large Vision-Language Models in the Korean Language
por: Kim, Yoonshik, et al.
Publicado: (2025)

Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks
por: Warner, Benjamin, et al.
Publicado: (2026)

Polishing Every Facet of the GEM: Testing Linguistic Competence of LLMs and Humans in Korean
por: Kim, SungHo, et al.
Publicado: (2025)

LPFQA: A Long-Tail Professional Forum-based Benchmark for LLM Evaluation
por: Zhu, Liya, et al.
Publicado: (2025)

HAE-RAE Bench: Evaluation of Korean Knowledge in Language Models
por: Son, Guijin, et al.
Publicado: (2023)

Ko-MuSR: A Multistep Soft Reasoning Benchmark for LLMs Capable of Understanding Korean
por: Park, Chanwoo, et al.
Publicado: (2025)

LLM-as-a-Judge & Reward Model: What They Can and Cannot Do
por: Son, Guijin, et al.
Publicado: (2024)

Benchmark Profiling: Mechanistic Diagnosis of LLM Benchmarks
por: Kim, Dongjun, et al.
Publicado: (2025)

K-EXAONE Technical Report
por: Choi, Eunbi, et al.
Publicado: (2026)

Evaluating Multimodal Generative AI with Korean Educational Standards
por: Park, Sanghee, et al.
Publicado: (2025)

DeFrame: Debiasing Large Language Models Against Framing Effects
por: Lim, Kahee, et al.
Publicado: (2026)

Evaluating Consistencies in LLM responses through a Semantic Clustering of Question Answering
por: Lee, Yanggyu, et al.
Publicado: (2024)

CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics
por: Somasekharan, Nithin, et al.
Publicado: (2025)