Cover Image

Assessing the Reliability of LLMs Annotations in the Context of Demographic Bias and Model Explanation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mohammadi, Hadi, Shahedi, Tina, Mosteiro, Pablo, Poesio, Massimo, Bagheri, Ayoub, Giachanou, Anastasia
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Explainability in Practice: A Survey of Explainable NLP Across Various Domains
by: Mohammadi, Hadi, et al.
Published: (2025)

Explainability-Based Token Replacement on LLM-Generated Text
by: Mohammadi, Hadi, et al.
Published: (2025)

EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Models
by: Mohammadi, Hadi, et al.
Published: (2025)

Evaluating GRPO and DPO for Faithful Chain-of-Thought Reasoning in LLMs
by: Mohammadi, Hadi, et al.
Published: (2025)

Improving Stance Detection by Leveraging Measurement Knowledge from Social Sciences: A Case Study of Dutch Political Tweets and Traditional Gender Role Division
by: Fang, Qixiang, et al.
Published: (2022)

Exploring Cultural Variations in Moral Judgments with Large Language Models
by: Mohammadi, Hadi, et al.
Published: (2025)

LLMs as mirrors of societal moral standards: reflection of cultural divergence and agreement across ethical topics
by: Meijer, Mijntje, et al.
Published: (2024)

Large Language Models as Mirrors of Societal Moral Standards
by: Papadopoulou, Evi, et al.
Published: (2024)

The Effectiveness of LLMs as Annotators: A Comparative Overview and Empirical Analysis of Direct Representation
by: Pavlovic, Maja, et al.
Published: (2024)

Do Large Language Models Understand Morality Across Cultures?
by: Mohammadi, Hadi, et al.
Published: (2025)

Large Language Models as Minecraft Agents
by: Madge, Chris, et al.
Published: (2024)

Grounded Misunderstandings in Asymmetric Dialogue: A Perspectivist Annotation Scheme for MapTask
by: Li, Nan, et al.
Published: (2025)

Can LLMs Detect Ambiguous Plural Reference? An Analysis of Split-Antecedent and Mereological Reference
by: Anh, Dang, et al.
Published: (2025)

Data Augmentation for Fake Reviews Detection in Multiple Languages and Multiple Domains
by: Liu, Ming, et al.
Published: (2025)

Integrating knowledge bases to improve coreference and bridging resolution for the chemical domain
by: Lu, Pengcheng, et al.
Published: (2024)

A LLM Benchmark based on the Minecraft Builder Dialog Agent Task
by: Madge, Chris, et al.
Published: (2024)

Understanding The Effect Of Temperature On Alignment With Human Opinions
by: Pavlovic, Maja, et al.
Published: (2024)

A Survey on Stereotype Detection in Natural Language Processing
by: Cignarella, Alessandra Teresa, et al.
Published: (2025)

Improving LLMs' Learning for Coreference Resolution
by: Gan, Yujian, et al.
Published: (2025)

LLMs Do Not See Age: Assessing Demographic Bias in Automated Systematic Review Synthesis
by: Aghaebe, Favour Yahdii, et al.
Published: (2025)

An Assessment of Human vs. Model Uncertainty in Soft-Label Learning and Calibration
by: Pavlovic, Maja, et al.
Published: (2026)

Referential ambiguity and clarification requests: comparing human and LLM behaviour
by: Madge, Chris, et al.
Published: (2025)

Elucidating Mechanisms of Demographic Bias in LLMs for Healthcare
by: Ahsan, Hiba, et al.
Published: (2025)

Which Demographics do LLMs Default to During Annotation?
by: Schäfer, Johannes, et al.
Published: (2024)

How and where does CLIP process negation?
by: Quantmeyer, Vincent, et al.
Published: (2024)

COBIAS: Assessing the Contextual Reliability of Bias Benchmarks for Language Models
by: Govil, Priyanshul, et al.
Published: (2024)

Finding Meaning in Embeddings: Concept Separation Curves
by: Keuren, Paul, et al.
Published: (2026)

ClarQ-LLM: A Benchmark for Models Clarifying and Requesting Information in Task-Oriented Dialog
by: Gan, Yujian, et al.
Published: (2024)

Human Label Variation in Implicit Discourse Relation Recognition
by: Yung, Frances, et al.
Published: (2026)

Extending Activation Steering to Broad Skills and Multiple Behaviours
by: van der Weij, Teun, et al.
Published: (2024)

ARTICLE: Annotator Reliability Through In-Context Learning
by: Dutta, Sujan, et al.
Published: (2024)

Local Explanations and Self-Explanations for Assessing Faithfulness in black-box LLMs
by: Fragkathoulas, Christos, et al.
Published: (2024)

Assessing the Reliability and Validity of GPT-4 in Annotating Emotion Appraisal Ratings
by: Ruder, Deniss, et al.
Published: (2025)

Modeling Annotator Disagreement with Demographic-Aware Experts and Synthetic Perspectives
by: Xu, Yinuo, et al.
Published: (2025)

Bias in, Bias out: Annotation Bias in Multilingual Large Language Models
by: Cui, Xia, et al.
Published: (2025)

Sometimes the Model doth Preach: Quantifying Religious Bias in Open LLMs through Demographic Analysis in Asian Nations
by: Shankar, Hari, et al.
Published: (2025)

Making Dialogue Grounding Data Rich: A Three-Tier Data Synthesis Framework for Generalized Referring Expression Comprehension
by: Shao, Juexi, et al.
Published: (2025)

Leveraging In-Context Learning for Political Bias Testing of LLMs
by: Haller, Patrick, et al.
Published: (2025)

Evaluating LLMs for Demographic-Targeted Social Bias Detection: A Comprehensive Benchmark Study
by: Majumdar, Ayan, et al.
Published: (2025)

Bias in the Ear of the Listener: Assessing Sensitivity in Audio Language Models Across Linguistic, Demographic, and Positional Variations
by: Wei, Sheng-Lun, et al.
Published: (2026)