Robust Evaluation Measures for Evaluating Social Biases in Masked Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Liu, Yang
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Measuring Social Biases in Masked Language Models by Proxy of Prediction Quality
by: Zalkikar, Rahul, et al.
Published: (2024)

Evaluating Short-Term Temporal Fluctuations of Social Biases in Social Media Data and Masked Language Models
by: Zhou, Yi, et al.
Published: (2024)

BiasCause: Evaluate Socially Biased Causal Reasoning of Large Language Models
by: Xie, Tian, et al.
Published: (2025)

Evaluating Large Language Model Biases in Persona-Steered Generation
by: Liu, Andy, et al.
Published: (2024)

Evaluating the Effect of Retrieval Augmentation on Social Biases
by: Zhang, Tianhui, et al.
Published: (2025)

FairPair: A Robust Evaluation of Biases in Language Models through Paired Perturbations
by: Dwivedi-Yu, Jane, et al.
Published: (2024)

This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Models
by: Li, Bryan, et al.
Published: (2023)

VisBias: Measuring Explicit and Implicit Social Biases in Vision Language Models
by: Huang, Jen-tse, et al.
Published: (2025)

Measuring Stereotype and Deviation Biases in Large Language Models
by: Wang, Daniel, et al.
Published: (2025)

Empirical Analysis of Decoding Biases in Masked Diffusion Models
by: Huang, Pengcheng, et al.
Published: (2025)

IndiBias: A Benchmark Dataset to Measure Social Biases in Language Models for Indian Context
by: Sahoo, Nihar Ranjan, et al.
Published: (2024)

Robust Infidelity: When Faithfulness Measures on Masked Language Models Are Misleading
by: Crothers, Evan, et al.
Published: (2023)

LLMs are Biased Evaluators But Not Biased for Retrieval Augmented Generation
by: Chen, Yen-Shan, et al.
Published: (2024)

Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs
by: Liu, Yang, et al.
Published: (2025)

Dutch CrowS-Pairs: Adapting a Challenge Dataset for Measuring Social Biases in Language Models for Dutch
by: Strazda, Elza, et al.
Published: (2025)

Generative Language Models Exhibit Social Identity Biases
by: Hu, Tiancheng, et al.
Published: (2023)

Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Models
by: Kumar, Abhishek, et al.
Published: (2024)

Estranged Predictions: Measuring Semantic Category Disruption with Masked Language Modelling
by: Liu, Yuxuan, et al.
Published: (2025)

The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models
by: Liu, Yan, et al.
Published: (2024)

Faithfulness Measurable Masked Language Models
by: Madsen, Andreas, et al.
Published: (2023)

Benchmarking Cognitive Biases in Large Language Models as Evaluators
by: Koo, Ryan, et al.
Published: (2023)

Are Bias Evaluation Methods Biased ?
by: Berrayana, Lina, et al.
Published: (2025)

Decoding the Mind of Large Language Models: A Quantitative Evaluation of Ideology and Biases
by: Hirose, Manari, et al.
Published: (2025)

BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked Language Models
by: Wiland, Jacek, et al.
Published: (2024)

SocialEval: Evaluating Social Intelligence of Large Language Models
by: Zhou, Jinfeng, et al.
Published: (2025)

Evaluating Biases in Context-Dependent Health Questions
by: Levy, Sharon, et al.
Published: (2024)

Evaluating Robustness of Large Language Models Against Multilingual Typographical Errors
by: Zhao, Raoyuan, et al.
Published: (2025)

Mitigating Social Biases in Language Models through Unlearning
by: Dige, Omkar, et al.
Published: (2024)

UnMASKed: Quantifying Gender Biases in Masked Language Models through Linguistically Informed Job Market Prompts
by: Parra, Iñigo
Published: (2024)

GPT-4 as Evaluator: Evaluating Large Language Models on Pest Management in Agriculture
by: Yang, Shanglong, et al.
Published: (2024)

Split and Merge: Aligning Position Biases in LLM-based Evaluators
by: Li, Zongjie, et al.
Published: (2023)

Breaking Bias, Building Bridges: Evaluation and Mitigation of Social Biases in LLMs via Contact Hypothesis
by: Raj, Chahat, et al.
Published: (2024)

Is It Bad to Work All the Time? Cross-Cultural Evaluation of Social Norm Biases in GPT-4
by: Liu, Zhuozhuo Joy, et al.
Published: (2025)

What is Your Favorite Gender, MLM? Gender Bias Evaluation in Multilingual Masked Language Models
by: Yu, Jeongrok, et al.
Published: (2024)

What am I missing here?: Evaluating Large Language Models for Masked Sentence Prediction
by: Wyatt, Charlie, et al.
Published: (2025)

Measuring the Robustness of Reference-Free Dialogue Evaluation Systems
by: Vasselli, Justin, et al.
Published: (2025)

Large Language Models Develop Novel Social Biases Through Adaptive Exploration
by: Wu, Addison J., et al.
Published: (2025)

Mitigating Biases in Language Models via Bias Unlearning
by: Liu, Dianqing, et al.
Published: (2025)

Exploring Bengali Religious Dialect Biases in Large Language Models with Evaluation Perspectives
by: Wasi, Azmine Toushik, et al.
Published: (2024)

ContrastScore: Towards Higher Quality, Less Biased, More Efficient Evaluation Metrics with Contrastive Evaluation
by: Wang, Xiao, et al.
Published: (2025)