Image de couverture de livre

BenchHub: A Unified Benchmark Suite for Holistic and Customizable LLM Evaluation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Kim, Eunsu, Yoo, Haneul, Son, Guijin, Patel, Hitesh, Agarwal, Amit, Oh, Alice
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for LLM Evaluation
par: Hong, Seokhee, et autres
Publié: (2025)

RoleConflictBench: A Benchmark of Role Conflict Scenarios for Evaluating LLMs' Contextual Sensitivity
par: Shin, Jisu, et autres
Publié: (2025)

LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation
par: Kim, Eunsu, et autres
Publié: (2024)

KoBBQ: Korean Bias Benchmark for Question Answering
par: Jin, Jiho, et autres
Publié: (2023)

AccessEval: Benchmarking Disability Bias in Large Language Models
par: Panda, Srikant, et autres
Publié: (2025)

The Generative AI Paradox on Evaluation: What It Can Solve, It May Not Evaluate
par: Oh, Juhyun, et autres
Publié: (2024)

Code-Switching In-Context Learning for Cross-Lingual Transfer of Large Language Models
par: Yoo, Haneul, et autres
Publié: (2025)

Revisiting the UID Hypothesis in LLM Reasoning Traces
par: Gwak, Minju, et autres
Publié: (2025)

Revisiting the Uniform Information Density Hypothesis in LLM Reasoning
par: Gwak, Minju, et autres
Publié: (2025)

Code-Switching Red-Teaming: LLM Evaluation for Safety and Multilingual Understanding
par: Yoo, Haneul, et autres
Publié: (2024)

Tokenization Matters: Improving Zero-Shot NER for Indic Languages
par: Pattnayak, Priyaranjan, et autres
Publié: (2025)

DREsS: Dataset for Rubric-based Essay Scoring on EFL Writing
par: Yoo, Haneul, et autres
Publié: (2024)

CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean
par: Kim, Eunsu, et autres
Publié: (2024)

Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation
par: Shin, Jisu, et autres
Publié: (2025)

Translating Hanja Historical Documents to Contemporary Korean and English
par: Son, Juhee, et autres
Publié: (2022)

LLM for Barcodes: Generating Diverse Synthetic Data for Identity Documents
par: Patel, Hitesh Laxmichand, et autres
Publié: (2024)

Code-Switching Curriculum Learning for Multilingual Transfer in LLMs
par: Yoo, Haneul, et autres
Publié: (2024)

MAQA: Evaluating Uncertainty Quantification in LLMs Regarding Data Uncertainty
par: Yang, Yongjin, et autres
Publié: (2024)

MUG-Eval: A Proxy Evaluation Framework for Multilingual Generation Capabilities in Any Language
par: Song, Seyoung, et autres
Publié: (2025)

BLUCK: A Benchmark Dataset for Bengali Linguistic Understanding and Cultural Knowledge
par: Kabir, Daeen, et autres
Publié: (2025)

Uncovering Factor Level Preferences to Improve Human-Model Alignment
par: Oh, Juhyun, et autres
Publié: (2024)

From National Curricula to Cultural Awareness: Constructing Open-Ended Culture-Specific Question Answering Dataset
par: Yoo, Haneul, et autres
Publié: (2026)

AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite
par: Bragg, Jonathan, et autres
Publié: (2025)

JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errors
par: Jin, Jiho, et autres
Publié: (2026)

NC-Bench: An LLM Benchmark for Evaluating Conversational Competence
par: Moore, Robert J., et autres
Publié: (2026)

Flex-TravelPlanner: A Benchmark for Flexible Planning with Language Agents
par: Oh, Juhyun, et autres
Publié: (2025)

LLM-Guided Lifecycle-Aware Clustering of Multi-Turn Customer Support Conversations
par: Pattnayak, Priyaranjan, et autres
Publié: (2026)

Redefining Evaluation Standards: A Unified Framework for Evaluating the Korean Capabilities of Language Models
par: Lee, Hanwool, et autres
Publié: (2025)

WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models
par: Tu, Shangqing, et autres
Publié: (2023)

Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents
par: Deng, Shihan, et autres
Publié: (2024)

Clinical QA 2.0: Multi-Task Learning for Answer Extraction and Categorization
par: Pattnayak, Priyaranjan, et autres
Publié: (2025)

Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks
par: Warner, Benjamin, et autres
Publié: (2026)

DAIQ: Auditing Demographic Attribute Inference from Question in LLMs
par: Panda, Srikant, et autres
Publié: (2025)

On the Effect of Uncertainty on Layer-wise Inference Dynamics
par: Kim, Sunwoo, et autres
Publié: (2025)

CricBench: A Multilingual Benchmark for Evaluating LLMs in Cricket Analytics
par: Agarwal, Parth, et autres
Publié: (2025)

Social Bias Benchmark for Generation: A Comparison of Generation and QA-Based Evaluations
par: Jin, Jiho, et autres
Publié: (2025)

OLA: Output Language Alignment in Code-Switched LLM Interactions
par: Oh, Juhyun, et autres
Publié: (2026)

GSM-SEM: Benchmark and Framework for Generating Semantically Variant Augmentations
par: Singh, Jyotika, et autres
Publié: (2026)

DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis
par: Patel, Liana, et autres
Publié: (2025)

JudgeBench: A Benchmark for Evaluating LLM-based Judges
par: Tan, Sijun, et autres
Publié: (2024)