Cover Image

Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targets

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Giorgi, Tommaso, Cima, Lorenzo, Fagni, Tiziano, Avvenuti, Marco, Cresci, Stefano
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Human-Computer Interaction
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The DSA Transparency Database: Auditing Self-reported Moderation Actions by Social Media
by: Trujillo, Amaury, et al.
Published: (2023)

Investigating the heterogenous effects of a massive content moderation intervention via Difference-in-Differences
by: Cima, Lorenzo, et al.
Published: (2024)

Contextualized Counterspeech: Strategies for Adaptation, Personalization, and Evaluation
by: Cima, Lorenzo, et al.
Published: (2024)

Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents
by: Morocho, Erika Elizabeth Taday, et al.
Published: (2026)

The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs
by: Calderon, Nitay, et al.
Published: (2025)

Blind Spots and Biases: Exploring the Role of Annotator Cognitive Biases in NLP
by: Gautam, Sanjana, et al.
Published: (2024)

Can Unconfident LLM Annotations Be Used for Confident Conclusions?
by: Gligorić, Kristina, et al.
Published: (2024)

MEGAnno+: A Human-LLM Collaborative Annotation System
by: Kim, Hannah, et al.
Published: (2024)

Emoji Reactions on Telegram: Unreliable Indicators of Emotional Resonance
by: Tardelli, Serena, et al.
Published: (2025)

Towards better social crisis data with HERMES: Hybrid sensing for EmeRgency ManagEment System
by: Avvenuti, Marco, et al.
Published: (2019)

Human-LLM Hybrid Text Answer Aggregation for Crowd Annotations
by: Li, Jiyi
Published: (2024)

Investigating Low-Cost LLM Annotation for~Spoken Dialogue Understanding Datasets
by: Druart, Lucas, et al.
Published: (2024)

BenchPress: A Human-in-the-Loop Annotation System for Rapid Text-to-SQL Benchmark Curation
by: Wenz, Fabian, et al.
Published: (2025)

Speech vs. Transcript: Does It Matter for Human Annotators in Speech Summarization?
by: Sharma, Roshan, et al.
Published: (2024)

QACP: An Annotated Question Answering Dataset for Assisting Chinese Python Programming Learners
by: Xiao, Rui, et al.
Published: (2024)

Unpacking Human Preference for LLMs: Demographically Aware Evaluation with the HUMAINE Framework
by: Petrova, Nora, et al.
Published: (2026)

A Comparative Study on Annotation Quality of Crowdsourcing and LLM via Label Aggregation
by: Li, Jiyi
Published: (2024)

Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement
by: Sarti, Gabriele, et al.
Published: (2025)

Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations
by: Hartmann, David, et al.
Published: (2025)

Efficient Models for the Detection of Hate, Abuse and Profanity
by: Tillmann, Christoph, et al.
Published: (2024)

If in a Crowdsourced Data Annotation Pipeline, a GPT-4
by: He, Zeyu, et al.
Published: (2024)

EEVEE: An Easy Annotation Tool for Natural Language Processing
by: Sorensen, Axel, et al.
Published: (2024)

Sandpiper: Orchestrated AI-Annotation for Educational Discourse at Scale
by: Hedley, Daryl, et al.
Published: (2026)

ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotation
by: Huang, Chen, et al.
Published: (2024)

Personas with Attitudes: Controlling LLMs for Diverse Data Annotation
by: Fröhling, Leon, et al.
Published: (2024)

Large Language Models and Thematic Analysis: Human-AI Synergy in Researching Hate Speech on Social Media
by: Breazu, Petre, et al.
Published: (2024)

Model-in-the-Loop (MILO): Accelerating Multimodal AI Data Annotation with LLMs
by: Wang, Yifan, et al.
Published: (2024)

Towards Human-like Multimodal Conversational Agent by Generating Engaging Speech
by: Kim, Taesoo, et al.
Published: (2025)

Humane Speech Synthesis through Zero-Shot Emotion and Disfluency Generation
by: Chaudhury, Rohan, et al.
Published: (2024)

Game Development as Human-LLM Interaction
by: Hong, Jiale, et al.
Published: (2024)

Text-to-SQL Domain Adaptation via Human-LLM Collaborative Data Annotation
by: Tian, Yuan, et al.
Published: (2025)

Socio-Emotional Response Generation: A Human Evaluation Protocol for LLM-Based Conversational Systems
by: Vanel, Lorraine, et al.
Published: (2024)

Dark Personality Traits and Online Toxicity: Linking Self-Reports to Reddit Activity
by: Cerulli, Aldo, et al.
Published: (2025)

Large Language Models Show Human-like Social Desirability Biases in Survey Responses
by: Salecha, Aadesh, et al.
Published: (2024)

A Close Reading Approach to Gender Narrative Biases in AI-Generated Stories
by: Raffini, Daniel, et al.
Published: (2025)

(Ir)rationality and Cognitive Biases in Large Language Models
by: Macmillan-Scott, Olivia, et al.
Published: (2024)

A Generalized LLM-Augmented BIM Framework: Application to a Speech-to-BIM system
by: Lee, Ghang, et al.
Published: (2024)

MEGnifying Emotion: Sentiment Analysis from Annotated Brain Data
by: Liu, Brian, et al.
Published: (2026)

MASCOT: Towards Multi-Agent Socio-Collaborative Companion Systems
by: Wang, Yiyang, et al.
Published: (2026)

Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AI
by: Guo, Hangzhi, et al.
Published: (2024)