Cover Image

In-Situ Behavioral Evaluation for LLM Fairness, Not Standardized-Test Scores

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tang, Zeyu, Truong, Sang T., Owens, Deonna, Sharma, Shreyas, Zhang, Yibo Jacky, Miranda, Brando, Koyejo, Sanmi
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence Computers and Society
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs
by: Wang, Angelina, et al.
Published: (2025)

CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
by: Vo, Truong, et al.
Published: (2025)

The Inadequacy of Offline LLM Evaluations: A Need to Account for Personalization in Model Behavior
by: Wang, Angelina, et al.
Published: (2025)

AI Evaluation Should Require Standardized Item-Level Data Releases
by: Jiang, Han, et al.
Published: (2026)

Why Do Safety Guardrails Degrade Across Languages?
by: Zhang, Max, et al.
Published: (2026)

Examining the Behavior of LLM Architectures Within the Framework of Standardized National Exams in Brazil
by: Locatelli, Marcelo Sartori, et al.
Published: (2024)

Reliable and Efficient Amortized Model-based Evaluation
by: Truong, Sang, et al.
Published: (2025)

Is Pre-training Truly Better Than Meta-Learning?
by: Miranda, Brando, et al.
Published: (2023)

A Multi-LLM Debiasing Framework
by: Owens, Deonna M., et al.
Published: (2024)

SpecEval: Evaluating Model Adherence to Behavior Specifications
by: Ahmed, Ahmed, et al.
Published: (2025)

TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots
by: Huang, Fangrui, et al.
Published: (2026)

Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track
by: Schaeffer, Rylan, et al.
Published: (2025)

Quantifying the Effect of Test Set Contamination on Generative Evaluations
by: Schaeffer, Rylan, et al.
Published: (2026)

Lean-ing on Quality: How High-Quality Data Beats Diverse Multilingual Data in AutoFormalization
by: Chan, Willy, et al.
Published: (2025)

Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data
by: Miranda, Brando, et al.
Published: (2023)

Is Backpropagation Optimal? When Synthetic Gradients Improve Sample Efficiency
by: Zhang, Yibo Jacky, et al.
Published: (2026)

Position: Beyond Sensitive Attributes, ML Fairness Should Quantify Structural Injustice via Social Determinants
by: Tang, Zeyu, et al.
Published: (2025)

The Sound of Syntax: Finetuning and Comprehensive Evaluation of Language Models for Speech Pathology
by: Patel, Fagun, et al.
Published: (2025)

Evaluating LLM Behavior in Hiring: Implicit Weights, Fairness Across Groups, and Alignment with Human Preferences
by: Hoffmann, Morgane, et al.
Published: (2026)

Crossing Linguistic Horizons: Finetuning and Comprehensive Evaluation of Vietnamese Large Language Models
by: Truong, Sang T., et al.
Published: (2024)

Quantifying the Importance of Data Alignment in Downstream Model Performance
by: Chawla, Krrish, et al.
Published: (2025)

On Fairness of Low-Rank Adaptation of Large Models
by: Ding, Zhoujie, et al.
Published: (2024)

ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment
by: Obbad, Elyas, et al.
Published: (2024)

Human or LLM as Standardized Patients? A Comparative Study for Medical Education
by: Zhang, Bingquan, et al.
Published: (2025)

Accept or Deny? Evaluating LLM Fairness and Performance in Loan Approval across Table-to-Text Serialization Approaches
by: Azime, Israel Abebe, et al.
Published: (2025)

The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies
by: Zhou, Jiaxu, et al.
Published: (2025)

Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learning
by: Hu, Jingyu, et al.
Published: (2024)

Discovering Implicit Large Language Model Alignment Objectives
by: Chen, Edward, et al.
Published: (2026)

Globalizing Fairness Attributes in Machine Learning: A Case Study on Health in Africa
by: Asiedu, Mercy Nyamewaa, et al.
Published: (2023)

Unveiling Scoring Processes: Dissecting the Differences between LLMs and Human Graders in Automatic Scoring
by: Wu, Xuansheng, et al.
Published: (2024)

The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awareness
by: Abdelnabi, Sahar, et al.
Published: (2025)

EvalCards: A Framework for Standardized Evaluation Reporting
by: Dhar, Ruchira, et al.
Published: (2025)

Agree to Disagree? A Meta-Evaluation of LLM Misgendering
by: Subramonian, Arjun, et al.
Published: (2025)

Collapsed Language Models Promote Fairness
by: Xu, Jingxuan, et al.
Published: (2024)

LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education
by: Weissburg, Iain, et al.
Published: (2024)

Are Models Trained on Indian Legal Data Fair?
by: Girhepuje, Sahil, et al.
Published: (2023)

RDBE: Reasoning Distillation-Based Evaluation Enhances Automatic Essay Scoring
by: Mohammadkhani, Ali Ghiasvand
Published: (2024)

Fantastic Bugs and Where to Find Them in AI Benchmarks
by: Truong, Sang, et al.
Published: (2025)

AI Cartography: Mapping the Latent Landscape of AI Benchmark Ecosystems
by: Hardy, Michael, et al.
Published: (2026)

Reasoning Models Don't Just Think Longer, They Move Differently
by: Gjølbye, Anders, et al.
Published: (2026)