Cover Image

Pushing the Boundaries of Multiple Choice Evaluation to One Hundred Options

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Nahyun, Son, Guijin
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

KAIO: A Collection of More Challenging Korean Questions
by: Lee, Nahyun, et al.
Published: (2025)

Controlling Language Confusion in Multilingual LLMs
by: Lee, Nahyun, et al.
Published: (2025)

KMMMU: Evaluation of Massive Multi-discipline Multimodal Understanding in Korean Language and Context
by: Lee, Nahyun, et al.
Published: (2026)

Multi-Task Inference: Can Large Language Models Follow Multiple Instructions at Once?
by: Son, Guijin, et al.
Published: (2024)

Ko-PIQA: A Korean Physical Commonsense Reasoning Dataset with Cultural Context
by: Choi, Dasol, et al.
Published: (2025)

Multi-Step Reasoning in Korean and the Emergent Mirage
by: Son, Guijin, et al.
Published: (2025)

Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gap
by: Ko, Hyunwoo, et al.
Published: (2025)

Wait, that's not an option: LLMs Robustness with Incorrect Multiple-Choice Options
by: Góral, Gracjan, et al.
Published: (2024)

Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought
by: Son, Guijin, et al.
Published: (2025)

Option-ID Based Elimination For Multiple Choice Questions
by: Zhu, Zhenhao, et al.
Published: (2025)

Revisiting the Uniform Information Density Hypothesis in LLM Reasoning
by: Gwak, Minju, et al.
Published: (2025)

Revisiting the UID Hypothesis in LLM Reasoning Traces
by: Gwak, Minju, et al.
Published: (2025)

LLM-as-a-Judge & Reward Model: What They Can and Cannot Do
by: Son, Guijin, et al.
Published: (2024)

From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for LLM Evaluation
by: Hong, Seokhee, et al.
Published: (2025)

Won: Establishing Best Practices for Korean Financial NLP
by: Son, Guijin, et al.
Published: (2025)

Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoning
by: Son, Guijin, et al.
Published: (2025)

It is Too Many Options: Pitfalls of Multiple-Choice Questions in Generative AI and Medical Education
by: Singh, Shrutika, et al.
Published: (2025)

HAE-RAE Bench: Evaluation of Korean Knowledge in Language Models
by: Son, Guijin, et al.
Published: (2023)

K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts
by: Lee, Nahyun, et al.
Published: (2026)

Redefining Evaluation Standards: A Unified Framework for Evaluating the Korean Capabilities of Language Models
by: Lee, Hanwool, et al.
Published: (2025)

Multiple-Choice Questions are Efficient and Robust LLM Evaluators
by: Zhang, Ziyin, et al.
Published: (2024)

One Script Instead of Hundreds? On Pretraining Romanized Encoder Language Models
by: Ebing, Benedikt, et al.
Published: (2026)

Judging What We Cannot Solve: A Consequence-Based Approach for Oracle-Free Evaluation of Research-Level Math
by: Son, Guijin, et al.
Published: (2026)

Self-Improving CAD Generation Agents with Finite Element Analysis as Feedback
by: Son, Guijin, et al.
Published: (2026)

Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation
by: Alhazmi, Elaf, et al.
Published: (2024)

ESG Classification by Implicit Rule Learning via GPT-4
by: Yun, Hyo Jeong, et al.
Published: (2024)

Improving Fine-grained Visual Understanding in VLMs through Text-Only Training
by: Choi, Dasol, et al.
Published: (2024)

BenchHub: A Unified Benchmark Suite for Holistic and Customizable LLM Evaluation
by: Kim, Eunsu, et al.
Published: (2025)

MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation
by: Lee, Dongjun, et al.
Published: (2024)

Improving Score Reliability of Multiple Choice Benchmarks with Consistency Evaluation and Altered Answer Choices
by: Cavalin, Paulo, et al.
Published: (2025)

ResearchMath-14K: Scaling Research-Level Mathematics via Agents
by: Son, Guijin, et al.
Published: (2026)

Simulating Training Data Leakage in Multiple-Choice Benchmarks for LLM Evaluation
by: Hidayat, Naila Shafirni, et al.
Published: (2025)

Beyond Multiple Choice: Evaluating Steering Vectors for Summarization
by: Braun, Joschka, et al.
Published: (2025)

Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation
by: Xu, Haoran, et al.
Published: (2024)

Generating Plausible Distractors for Multiple-Choice Questions via Student Choice Prediction
by: Lee, Yooseop, et al.
Published: (2025)

Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Above
by: Balepur, Nishant, et al.
Published: (2025)

Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration
by: Sun, Qiushi, et al.
Published: (2023)

Math Multiple Choice Question Generation via Human-Large Language Model Collaboration
by: Lee, Jaewook, et al.
Published: (2024)

Differentiating Choices via Commonality for Multiple-Choice Question Answering
by: Deng, Wenqing, et al.
Published: (2024)

Right Answer, Wrong Score: Uncovering the Inconsistencies of LLM Evaluation in Multiple-Choice Question Answering
by: Molfese, Francesco Maria, et al.
Published: (2025)