Imagen de Portada

OKBench: Democratizing LLM Evaluation with Fully Automated, On-Demand, Open Knowledge Benchmarking

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Li, Yanhong, Xu, Tianyang, Tang, Kenan, Livescu, Karen, McAllester, David, Zhou, Jiawei
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Context-Efficient Retrieval with Factual Decomposition
por: Li, Yanhong, et al.
Publicado: (2025)

Training Transformers as a Universal Computer
por: Xu, Ruize, et al.
Publicado: (2026)

Chunk-Distilled Language Modeling
por: Li, Yanhong, et al.
Publicado: (2024)

On the Predictive Power of Representation Dispersion in Language Models
por: Li, Yanhong, et al.
Publicado: (2025)

PENCIL: Long Thoughts with Short Memory
por: Yang, Chenxiao, et al.
Publicado: (2025)

Cross-Modal Taxonomic Generalization in (Vision-) Language Models
por: Xu, Tianyang, et al.
Publicado: (2026)

OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Data
por: Du, Yuwen, et al.
Publicado: (2026)

DIY-MKG: An LLM-Based Polyglot Language Learning System
por: Tang, Kenan, et al.
Publicado: (2025)

Flow-SLM: Joint Learning of Linguistic and Acoustic Information for Spoken Language Modeling
por: Chou, Ju-Chieh, et al.
Publicado: (2025)

AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agents
por: Tang, Jiabin, et al.
Publicado: (2025)

Text or Pixels? It Takes Half: On the Token Efficiency of Visual Text Inputs in Multimodal LLMs
por: Li, Yanhong, et al.
Publicado: (2025)

The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
por: Lu, Chris, et al.
Publicado: (2024)

JudgeAgent: Beyond Static Benchmarks for Knowledge-Driven and Dynamic LLM Evaluation
por: Shi, Zhichao, et al.
Publicado: (2025)

ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM
por: Su, Zhaochen, et al.
Publicado: (2024)

DSGBench: A Diverse Strategic Game Benchmark for Evaluating LLM-based Agents in Complex Decision-Making Environments
por: Tang, Wenjie, et al.
Publicado: (2025)

Open-LLM-Leaderboard: From Multi-choice to Open-style Questions for LLMs Evaluation, Benchmark, and Arena
por: Myrzakhan, Aidar, et al.
Publicado: (2024)

Steamroller Problems: An Evaluation of LLM Reasoning Capability with Automated Theorem Prover Strategies
por: McGinness, Lachlan, et al.
Publicado: (2024)

When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation
por: Xu, Wenda, et al.
Publicado: (2025)

C-FAITH: A Chinese Fine-Grained Benchmark for Automated Hallucination Evaluation
por: Zhang, Xu, et al.
Publicado: (2025)

SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scale
por: Gueuwou, Shester, et al.
Publicado: (2024)

Structured Tree Alignment for Evaluation of (Speech) Constituency Parsing
por: Shi, Freda, et al.
Publicado: (2024)

EventCast: Hybrid Demand Forecasting in E-Commerce with LLM-Based Event Knowledge
por: Hu, Congcong, et al.
Publicado: (2026)

KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs
por: Markowitz, Elan, et al.
Publicado: (2025)

Assessing Bias in Metric Models for LLM Open-Ended Generation Bias Benchmarks
por: Demchak, Nathaniel, et al.
Publicado: (2024)

LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs
por: LLM-jp, et al.
Publicado: (2024)

Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmark
por: Park, Chanjun, et al.
Publicado: (2024)

Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs
por: Liu, Xiaoze, et al.
Publicado: (2024)

The Automated LLM Speedrunning Benchmark: Reproducing NanoGPT Improvements
por: Zhao, Bingchen, et al.
Publicado: (2025)

Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks
por: Li, Peiyu, et al.
Publicado: (2025)

SHIELD: Evaluation and Defense Strategies for Copyright Compliance in LLM Text Generation
por: Liu, Xiaoze, et al.
Publicado: (2024)

Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Models
por: Wong, Lionel, et al.
Publicado: (2025)

Big Reasoning with Small Models: Instruction Retrieval at Inference Time
por: Alkiek, Kenan, et al.
Publicado: (2025)

Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction
por: Qi, Ji, et al.
Publicado: (2023)

Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method
por: Zhao, Tianzhe, et al.
Publicado: (2026)

Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents
por: Deng, Shihan, et al.
Publicado: (2024)

IRLBench: A Multi-modal, Culturally Grounded, Parallel Irish-English Benchmark for Open-Ended LLM Reasoning Evaluation
por: Tran, Khanh-Tung, et al.
Publicado: (2025)

KokushiMD-10: Benchmark for Evaluating Large Language Models on Ten Japanese National Healthcare Licensing Examinations
por: Liu, Junyu, et al.
Publicado: (2025)

Who Benchmarks the Benchmarks? A Case Study of LLM Evaluation in Icelandic
por: Ingimundarson, Finnur Ágúst, et al.
Publicado: (2026)

Apertus: Democratizing Open and Compliant LLMs for Global Language Environments
por: Apertus, Project, et al.
Publicado: (2025)

BenchGuard: Who Guards the Benchmarks? Automated Auditing of LLM Agent Benchmarks
por: Tu, Xinming, et al.
Publicado: (2026)