Evaluating LLMs for Demographic-Targeted Social Bias Detection: A Comprehensive Benchmark Study

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Majumdar, Ayan, Chen, Feihao, Li, Jinghui, Wang, Xiaozhen
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Computers and Society Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Leveraging Prototypical Representations for Mitigating Social Bias without Demographic Information
by: Iskander, Shadi, et al.
Published: (2024)

Who Gets Which Message? Auditing Demographic Bias in LLM-Generated Targeted Text
by: Islam, Tunazzina
Published: (2026)

GG-BBQ: German Gender Bias Benchmark for Question Answering
by: Satheesh, Shalaka, et al.
Published: (2025)

Template-Based Probes Are Imperfect Lenses for Counterfactual Bias Evaluation in LLMs
by: Kohankhaki, Farnaz, et al.
Published: (2024)

GRILE: A Benchmark for Grammar Reasoning and Explanation in Romanian LLMs
by: Dumitran, Adrian-Marius, et al.
Published: (2025)

BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responses
by: Xu, Xin, et al.
Published: (2025)

PakBBQ: A Culturally Adapted Bias Benchmark for QA
by: Hashmat, Abdullah, et al.
Published: (2025)

Evaluating the Promise and Pitfalls of LLMs in Hiring Decisions
by: Anzenberg, Eitan, et al.
Published: (2025)

RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?
by: de Wynter, Adrian, et al.
Published: (2024)

CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models
by: Xia, Peng, et al.
Published: (2024)

DSO: Direct Steering Optimization for Bias Mitigation
by: Paes, Lucas Monteiro, et al.
Published: (2025)

Toward Automated Detection of Biased Social Signals from the Content of Clinical Conversations
by: Chen, Feng, et al.
Published: (2024)

ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming
by: Tedeschi, Simone, et al.
Published: (2024)

CodeMirage: A Multi-Lingual Benchmark for Detecting AI-Generated and Paraphrased Source Code from Production-Level LLMs
by: Guo, Hanxi, et al.
Published: (2025)

AppellateGen: A Benchmark for Appellate Legal Judgment Generation
by: Yang, Hongkun, et al.
Published: (2026)

Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs
by: Chen, Yen-Shan, et al.
Published: (2026)

Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation
by: Vargas, Francisco, et al.
Published: (2020)

Deep Learning Approaches for Detecting Adversarial Cyberbullying and Hate Speech in Social Networks
by: Azumah, Sylvia Worlali, et al.
Published: (2024)

Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Risks
by: Chen, Sirui, et al.
Published: (2025)

LatentQA: Teaching LLMs to Decode Activations Into Natural Language
by: Pan, Alexander, et al.
Published: (2024)

Fair Representation in Parliamentary Summaries: Measuring and Mitigating Inclusion Bias
by: Cunningham, Eoghan, et al.
Published: (2025)

Benchmarking the Legal Reasoning of LLMs in Arabic Islamic Inheritance Cases
by: AlDahoul, Nouar, et al.
Published: (2025)

Mitigating Bias for Question Answering Models by Tracking Bias Influence
by: Ma, Mingyu Derek, et al.
Published: (2023)

LIBRA: Measuring Bias of Large Language Model from a Local Context
by: Pang, Bo, et al.
Published: (2025)

ELMES: An Automated Framework for Evaluating Large Language Models in Educational Scenarios
by: Wei, Shou'ang, et al.
Published: (2025)

Intrinsic Meets Extrinsic Fairness: Assessing the Downstream Impact of Bias Mitigation in Large Language Models
by: Arzaghi', 'Mina, et al.
Published: (2025)

MateInfoUB: A Real-World Benchmark for Testing LLMs in Competitive, Multilingual, and Multimodal Educational Tasks
by: Marius, Dumitran Adrian, et al.
Published: (2025)

Towards Robust Evaluation of Unlearning in LLMs via Data Transformations
by: Joshi, Abhinav, et al.
Published: (2024)

Identifying Climate Targets in National Laws and Policies using Machine Learning
by: Juhasz, Matyas, et al.
Published: (2024)

TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law
by: Hui, Zheng, et al.
Published: (2025)

Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Models
by: Kumar, Abhishek, et al.
Published: (2024)

CAMEL-Bench: A Comprehensive Arabic LMM Benchmark
by: Ghaboura, Sara, et al.
Published: (2024)

Evaluating Prompt Engineering Techniques for Accuracy and Confidence Elicitation in Medical LLMs
by: Naderi, Nariman, et al.
Published: (2025)

Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs
by: Scarlatos, Alexander, et al.
Published: (2024)

Representation Bias of Adolescents in AI: A Bilingual, Bicultural Study
by: Wolfe, Robert, et al.
Published: (2024)

Bias and Fairness in Large Language Models: A Survey
by: Gallegos, Isabel O., et al.
Published: (2023)

Evaluating Reasoning LLMs for Suicide Screening with the Columbia-Suicide Severity Rating Scale
by: Patil, Avinash, et al.
Published: (2025)

Toward Cultural Interpretability: A Linguistic Anthropological Framework for Describing and Evaluating Large Language Models (LLMs)
by: Jones, Graham M., et al.
Published: (2024)

The ProLiFIC dataset: Leveraging LLMs to Unveil the Italian Lawmaking Process
by: Contestabile, Matilde, et al.
Published: (2025)

Exploring the Potential of the Large Language Models (LLMs) in Identifying Misleading News Headlines
by: Rony, Md Main Uddin, et al.
Published: (2024)