Cover Image

From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Tianle, Chiang, Wei-Lin, Frick, Evan, Dunlap, Lisa, Wu, Tianhao, Zhu, Banghua, Gonzalez, Joseph E., Stoica, Ion
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

How to Evaluate Reward Models for RLHF
by: Frick, Evan, et al.
Published: (2024)

Prompt-to-Leaderboard
by: Frick, Evan, et al.
Published: (2025)

VisionArena: 230K Real World User-VLM Conversations with Preference Labels
by: Chou, Christopher, et al.
Published: (2024)

Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
by: Chiang, Wei-Lin, et al.
Published: (2024)

OR-Bench: An Over-Refusal Benchmark for Large Language Models
by: Cui, Justin, et al.
Published: (2024)

RouteLLM: Learning to Route LLMs with Preference Data
by: Ong, Isaac, et al.
Published: (2024)

Some Present-Day Problems of Romanian Library Science
by: Stoica, Ion
Published: (1973)

The Central University Library, Bucharest. Over Seventy-five Years in the History of a Collection
by: Stoica, Ion
Published: (1972)

CRS Arena: Crowdsourced Benchmarking of Conversational Recommender Systems
by: Bernard, Nolwenn, et al.
Published: (2024)

Fairness in Serving Large Language Models
by: Sheng, Ying, et al.
Published: (2023)

Copilot Arena: A Platform for Code LLM Evaluation in the Wild
by: Chi, Wayne, et al.
Published: (2025)

DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis
by: Patel, Liana, et al.
Published: (2025)

Adversarial Arena: Crowdsourcing Data Generation through Interactive Competition
by: Goyal, Prasoon, et al.
Published: (2026)

JudgeBench: A Benchmark for Evaluating LLM-based Judges
by: Tan, Sijun, et al.
Published: (2024)

BuilderBench: The Building Blocks of Intelligent Agents
by: Ghugare, Raj, et al.
Published: (2025)

GameArena: Evaluating LLM Reasoning through Live Computer Games
by: Hu, Lanxiang, et al.
Published: (2024)

Viscous shock fluctuations in KPZ
by: Dunlap, Alexander, et al.
Published: (2024)

Invariant measures and shocks in the KPZ fixed point
by: Dunlap, Alexander, et al.
Published: (2025)

If in a Crowdsourced Data Annotation Pipeline, a GPT-4
by: He, Zeyu, et al.
Published: (2024)

MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation Systems
by: Thakur, Nandan, et al.
Published: (2024)

Benevolent Builder: Appraising Andrew Carnegie.
by: Deitch, Joseph
Published: (1984)

DPImageBench: A Unified Benchmark for Differentially Private Image Synthesis
by: Gong, Chen, et al.
Published: (2025)

Search Arena: Analyzing Search-Augmented LLMs
by: Miroyan, Mihran, et al.
Published: (2025)

Speech Foundation Models and Crowdsourcing for Efficient, High-Quality Data Collection
by: Lee, Beomseok, et al.
Published: (2024)

EDIT-Bench: Evaluating LLM Abilities to Perform Real-World Instructed Code Edits
by: Chi, Wayne, et al.
Published: (2025)

Data Quality in Crowdsourcing and Spamming Behavior Detection
by: Ba, Yang, et al.
Published: (2024)

Qrita: High-performance Top-k and Top-p using Pivot-based Truncation and Selection
by: Park, Jongseok, et al.
Published: (2026)

K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model
by: Cao, Shiyi, et al.
Published: (2026)

OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value
by: Cai, Mengzhang, et al.
Published: (2025)

ELT-Bench: An End-to-End Benchmark for Evaluating AI Agents on ELT Pipelines
by: Jin, Tengjun, et al.
Published: (2025)

Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF
by: Zhu, Banghua, et al.
Published: (2024)

KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakes
by: Lai, Eugenie, et al.
Published: (2025)

Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity
by: Griggs, Tyler, et al.
Published: (2024)

TopoBench: Benchmarking LLMs on Hard Topological Reasoning
by: Maniparambil, Mayug, et al.
Published: (2026)

S-LoRA: Serving Thousands of Concurrent LoRA Adapters
by: Sheng, Ying, et al.
Published: (2023)

Trustless Audits without Revealing Data or Models
by: Waiwitlikhit, Suppakit, et al.
Published: (2024)

M$^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling
by: Mishra, Mayank, et al.
Published: (2026)

Mobile Coverage Analysis using Crowdsourced Data
by: Wong, Timothy, et al.
Published: (2025)

Rapid data quality investigations of gravitational-wave events with the Data Quality Report Builder toolkit
by: Davis, Derek, et al.
Published: (2026)

From Easy to Hard++: Promoting Differentially Private Image Synthesis Through Spatial-Frequency Curriculum
by: Gong, Chen, et al.
Published: (2026)