Image de couverture de livre

Toward an Evaluation Science for Generative AI Systems

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Weidinger, Laura, Raji, Inioluwa Deborah, Wallach, Hanna, Mitchell, Margaret, Wang, Angelina, Salaudeen, Olawale, Bommasani, Rishi, Ganguli, Deep, Koyejo, Sanmi, Isaac, William
Format:	Preprint
Publié:	2025
Sujets:	Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Causally Inspired Regularization Enables Domain General Representations
par: Salaudeen, Olawale, et autres
Publié: (2024)

Are Domain Generalization Benchmarks with Accuracy on the Line Misspecified?
par: Salaudeen, Olawale, et autres
Publié: (2025)

Measurement to Meaning: A Validity-Centered Framework for AI Evaluation
par: Salaudeen, Olawale, et autres
Publié: (2025)

Concrete Problems in AI Safety, Revisited
par: Raji, Inioluwa Deborah, et autres
Publié: (2023)

Evaluating Prediction-based Interventions with Human Decision Makers In Mind
par: Raji, Inioluwa Deborah, et autres
Publié: (2025)

Multi-lingual Functional Evaluation for Large Language Models
par: Ojewale, Victor, et autres
Publié: (2025)

NeurIPS should lead scientific consensus on AI policy
par: Bommasani, Rishi
Publié: (2025)

The Societal Impact of Foundation Models: Advancing Evidence-based AI Policy
par: Bommasani, Rishi
Publié: (2025)

Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
par: Ojewale, Victor, et autres
Publié: (2024)

"I Searched for a Religious Song in Amharic and Got Sexual Content Instead": Investigating Online Harm in Low-Resourced Languages on YouTube
par: Nigatu, Hellina Hailu, et autres
Publié: (2024)

ImageNot: A contrast with ImageNet preserves model rankings
par: Salaudeen, Olawale, et autres
Publié: (2024)

The Inadequacy of Offline LLM Evaluations: A Need to Account for Personalization in Model Behavior
par: Wang, Angelina, et autres
Publié: (2025)

Let's Measure Information Step-by-Step: AI-Based Evaluation Beyond Vibes
par: Robertson, Zachary, et autres
Publié: (2025)

SCENEBench: An Audio Understanding Benchmark Grounded in Assistive and Industrial Use Cases
par: Iyer, Laya, et autres
Publié: (2026)

From Individual Experience to Collective Evidence: A Reporting-Based Framework for Identifying Systemic Harms
par: Dai, Jessica, et autres
Publié: (2025)

AI auditing: The Broken Bus on the Road to AI Accountability
par: Birhane, Abeba, et autres
Publié: (2024)

Aggregated Individual Reporting for Post-Deployment Evaluation
par: Dai, Jessica, et autres
Publié: (2025)

CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
par: Vo, Truong, et autres
Publié: (2025)

Proxy Methods for Domain Adaptation
par: Tsai, Katherine, et autres
Publié: (2024)

Position: Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead
par: Sühr, Tom, et autres
Publié: (2025)

The Data Addition Dilemma
par: Shen, Judy Hanwen, et autres
Publié: (2024)

Trustworthy Social Bias Measurement
par: Bommasani, Rishi, et autres
Publié: (2022)

Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs
par: Wang, Angelina, et autres
Publié: (2025)

Disclosure and Evaluation as Fairness Interventions for General-Purpose AI
par: Raman, Vyoma, et autres
Publié: (2025)

The Reality of AI and Biorisk
par: Peppin, Aidan, et autres
Publié: (2024)

A Framework for Objective-Driven Dynamical Stochastic Fields
par: Zhang, Yibo Jacky, et autres
Publié: (2025)

Do AI Companies Make Good on Voluntary Commitments to the White House?
par: Wang, Jennifer, et autres
Publié: (2025)

The Optimization Paradox in Clinical AI Multi-Agent Systems
par: Bedi, Suhana, et autres
Publié: (2025)

Pretraining Scaling Laws for Generative Evaluations of Language Models
par: Schaeffer, Rylan, et autres
Publié: (2025)

Stop Automating Peer Review Without Rigorous Evaluation
par: Baumann, Joachim, et autres
Publié: (2026)

Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations
par: Salaudeen, Olawale, et autres
Publié: (2025)

What's in a Query: Polarity-Aware Distribution-Based Fair Ranking
par: Balagopalan, Aparna, et autres
Publié: (2025)

Effective Mitigations for Systemic Risks from General-Purpose AI
par: Uuk, Risto, et autres
Publié: (2024)

Advancing Science- and Evidence-based AI Policy
par: Bommasani, Rishi, et autres
Publié: (2025)

Aligning Compound AI Systems via System-level DPO
par: Wang, Xiangwen, et autres
Publié: (2025)

In-Context Learning of Energy Functions
par: Schaeffer, Rylan, et autres
Publié: (2024)

Discovering Implicit Large Language Model Alignment Objectives
par: Chen, Edward, et autres
Publié: (2026)

High-Dimensional Markov-switching Ordinary Differential Processes
par: Tsai, Katherine, et autres
Publié: (2024)

Distributional Machine Unlearning via Selective Data Removal
par: Allouah, Youssef, et autres
Publié: (2025)

HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance
par: Zhu, Junzhe, et autres
Publié: (2023)