Cover Image

Mapping AI Benchmark Data to Quantitative Risk Estimates Through Expert Elicitation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Murray, Malcolm, Papadatos, Henry, Quarks, Otter, Gimenez, Pierre-François, Campos, Simeon
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Frontier AI Risk Management Framework: Bridging the Gap Between Current AI Practices and Established Risk Management
by: Campos, Simeon, et al.
Published: (2025)

A Methodology for Quantitative AI Risk Modeling
by: Murray, Malcolm, et al.
Published: (2025)

The Role of Risk Modeling in Advanced AI Risk Management
by: Touzet, Chloé, et al.
Published: (2025)

Linear Probe Penalties Reduce LLM Sycophancy
by: Papadatos, Henry, et al.
Published: (2024)

Evaluating the Goal-Directedness of Large Language Models
by: Everitt, Tom, et al.
Published: (2025)

Toward Quantitative Modeling of Cybersecurity Risks Due to AI Misuse
by: Barrett, Steve, et al.
Published: (2025)

Evaluating AI Providers' Frontier Safety Frameworks
by: Stelling, Lily, et al.
Published: (2025)

Quantitative AI Risk Assessments: Opportunities and Challenges
by: Piorkowski, David, et al.
Published: (2022)

Certifiably robust malware detectors by design
by: Gimenez, Pierre-Francois, et al.
Published: (2025)

Mapping Industry Practices to the EU AI Act's GPAI Code of Practice Safety and Security Measures
by: Stelling, Lily, et al.
Published: (2025)

Can LLMs Assist Expert Elicitation for Probabilistic Causal Modeling?
by: Shaposhnyk, Olha, et al.
Published: (2025)

AI Data Readiness Inspector (AIDRIN) for Quantitative Assessment of Data Readiness for AI
by: Hiniduma, Kaveen, et al.
Published: (2024)

Data Therapist: Eliciting Domain Knowledge from Subject Matter Experts Using Large Language Models
by: Shin, Sungbok, et al.
Published: (2025)

Evaluating the Capability of Video Question Generation for Expert Knowledge Elicitation
by: Zhang, Huaying, et al.
Published: (2025)

Where are the Humans? A Scoping Review of Fairness in Multi-agent AI Systems
by: Allmendinger, Simeon, et al.
Published: (2026)

Open Problems in Frontier AI Risk Management
by: Ziosi, Marta, et al.
Published: (2026)

Engaged AI Governance: Addressing the Last Mile Challenge Through Internal Expert Collaboration
by: Jarvers, Simon, et al.
Published: (2026)

Mapping AI Risk Mitigations: Evidence Scan and Preliminary AI Risk Mitigation Taxonomy
by: Saeri, Alexander K., et al.
Published: (2025)

BC Protocol: Structured Dual-Expert Dialogue for Eliciting High-Quality Chain-of-Thought Post-Training Data
by: Zou, Bo, et al.
Published: (2026)

AIDABench: AI Data Analytics Benchmark
by: Yang, Yibo, et al.
Published: (2026)

Detecting Benchmark Contamination Through Watermarking
by: Sander, Tom, et al.
Published: (2025)

Unsupervised Elicitation of Language Models
by: Wen, Jiaxin, et al.
Published: (2025)

AI Cartography: Mapping the Latent Landscape of AI Benchmark Ecosystems
by: Hardy, Michael, et al.
Published: (2026)

The Elicitation Game: Evaluating Capability Elicitation Techniques
by: Hofstätter, Felix, et al.
Published: (2025)

MapQA: Open-domain Geospatial Question Answering on Map Data
by: Li, Zekun, et al.
Published: (2025)

Affirmative safety: An approach to risk management for high-risk AI
by: Wasil, Akash R., et al.
Published: (2024)

Dark Speculation: Combining Qualitative and Quantitative Understanding in Frontier AI Risk Analysis
by: Carpenter, Daniel, et al.
Published: (2025)

Combining Confidence Elicitation and Sample-based Methods for Uncertainty Quantification in Misinformation Mitigation
by: Rivera, Mauricio, et al.
Published: (2024)

A Mobile Application Front-End for Presenting Explainable AI Results in Diabetes Risk Estimation
by: Willson, Bernardus, et al.
Published: (2025)

InsightEval: An Expert-Curated Benchmark for Assessing Insight Discovery in LLM-Driven Data Agents
by: Zhu, Zhenghao, et al.
Published: (2025)

QuantBench: Benchmarking AI Methods for Quantitative Investment
by: Wang, Saizhuo, et al.
Published: (2025)

Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Data
by: Liu, Xiao, et al.
Published: (2024)

Normative Common Ground Replication (NormCoRe): Replication-by-Translation for Studying Norms in Multi-Agent AI
by: Deck, Luca, et al.
Published: (2026)

BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices
by: Reuel, Anka, et al.
Published: (2024)

CMT-Benchmark: A Benchmark for Condensed Matter Theory Built by Expert Researchers
by: Pan, Haining, et al.
Published: (2025)

MindSearch: Mimicking Human Minds Elicits Deep AI Searcher
by: Chen, Zehui, et al.
Published: (2024)

Eliciting In-Context Learning in Vision-Language Models for Videos Through Curated Data Distributional Properties
by: Yu, Keunwoo Peter, et al.
Published: (2023)

RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models
by: Wang, Zekun Moore, et al.
Published: (2023)

ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
by: Chollet, Francois, et al.
Published: (2025)

Why do Experts Disagree on Existential Risk and P(doom)? A Survey of AI Experts
by: Field, Severin
Published: (2025)