Cover Image

Who and What? Using Linguistic Features and Annotator Characteristics to Analyze Annotation Variation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Maurer, Maximilian, Linde, Maximilian, Lapesa, Gabriella
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Computers and Society
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Toeing the Party Line: Election Manifestos as a Key to Understand Political Discourse on Twitter
by: Maurer, Maximilian, et al.
Published: (2024)

Towards a Perspectivist Turn in Argument Quality Assessment
by: Romberg, Julia, et al.
Published: (2025)

Tell Me What You Know About Sexism: Expert-LLM Interaction Strategies and Co-Created Definitions for Zero-Shot Sexism Detection
by: Reuver, Myrthe, et al.
Published: (2025)

A Modular Taxonomy for Hate Speech Definitions and Its Impact on Zero-Shot LLM Classification Performance
by: Melis, Matteo, et al.
Published: (2025)

Navigating the Risks of Using Large Language Models for Text Annotation in Social Science Research
by: Lin, Hao, et al.
Published: (2025)

An Annotated Reading of 'The Singer of Tales' in the LLM Era
by: Varshney, Kush R.
Published: (2025)

Measuring Large Language Models Capacity to Annotate Journalistic Sourcing
by: Vincent, Subramaniam, et al.
Published: (2024)

What is Stigma Attributed to? A Theory-Grounded, Expert-Annotated Interview Corpus for Demystifying Mental-Health Stigma
by: Meng, Han, et al.
Published: (2025)

From Emotion to Expression: Theoretical Foundations and Resources for Fear Speech
by: Shankaran, Vigneshwaran, et al.
Published: (2026)

Investigating Subjective Factors of Argument Strength: Storytelling, Emotions, and Hedging
by: Quensel, Carlotta, et al.
Published: (2025)

Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas
by: Stähelin, Lukas, et al.
Published: (2026)

SPOT: An Annotated French Corpus and Benchmark for Detecting Critical Interventions in Online Conversations
by: Berriche, Manon, et al.
Published: (2025)

Large Models of What? Mistaking Engineering Achievements for Human Linguistic Agency
by: Birhane, Abeba, et al.
Published: (2024)

WhatsApp Vaccine Discourse (WhaVax): An Expert-Annotated Dataset and Benchmark for Health Misinformation Detection
by: Santos, Jônatas H. dos, et al.
Published: (2026)

IDEAlign: Comparing Large Language Models to Human Experts in Open-ended Interpretive Annotations
by: Nam, Hyunji, et al.
Published: (2025)

Three Models of RLHF Annotation: Extension, Evidence, and Authority
by: Coyne, Steve
Published: (2026)

GPT is Not an Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction
by: Felkner, Virginia K., et al.
Published: (2024)

Don't Blame the Data, Blame the Model: Understanding Noise and Bias When Learning from Subjective Annotations
by: Anand, Abhishek, et al.
Published: (2024)

The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation
by: Munir, Sheza, et al.
Published: (2026)

Who Decides What Is Harmful? Content Moderation Policy Through A Multi-Agent Personalised Inference Framework
by: Gajewska, Ewelina, et al.
Published: (2026)

Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models
by: Abraham, Louis, et al.
Published: (2024)

Augmenting Human-Annotated Training Data with Large Language Model Generation and Distillation in Open-Response Assessment
by: Borchers, Conrad, et al.
Published: (2025)

Automating Governing Knowledge Commons and Contextual Integrity (GKC-CI) Privacy Policy Annotations with Large Language Models
by: Chanenson, Jake, et al.
Published: (2023)

QSTN: A Modular Framework for Robust Questionnaire Inference with Large Language Models
by: Kreutner, Maximilian, et al.
Published: (2025)

Analyzing Dataset Annotation Quality Management in the Wild
by: Klie, Jan-Christoph, et al.
Published: (2023)

KPoEM: A Human-Annotated Dataset for Emotion Classification and RAG-Based Poetry Generation in Korean Modern Poetry
by: Lim, Iro, et al.
Published: (2025)

Evaluating Large Language Models Against Human Annotators in Latent Content Analysis: Sentiment, Political Leaning, Emotional Intensity, and Sarcasm
by: Bojic, Ljubisa, et al.
Published: (2025)

CAPC-CG: A Large-Scale, Expert-Directed LLM-Annotated Corpus of Adaptive Policy Communication in China
by: Sun, Bolun, et al.
Published: (2025)

Analyzing Fairness in Deepfake Detection With Massively Annotated Databases
by: Xu, Ying, et al.
Published: (2022)

Linguistic Uncertainty and Engagement in Arabic-Language X (formerly Twitter) Discourse
by: Soufan, Mohamed
Published: (2026)

SWAY: A Counterfactual Computational Linguistic Approach to Measuring and Mitigating Sycophancy
by: Bhalla, Joy, et al.
Published: (2026)

Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination
by: Fleisig, Eve, et al.
Published: (2024)

Evaluation is all you need. Prompting Generative Large Language Models for Annotation Tasks in the Social Sciences. A Primer using Open Models
by: Weber, Maximilian, et al.
Published: (2023)

"What's Up, Doc?": Analyzing How Users Seek Health Information in Large-Scale Conversational AI Datasets
by: Paruchuri, Akshay, et al.
Published: (2025)

Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models
by: Girrbach, Leander, et al.
Published: (2025)

Rethinking Suicidal Ideation Detection: A Trustworthy Annotation Framework and Cross-Lingual Model Evaluation
by: Dzafic, Amina, et al.
Published: (2025)

Analyzing the Performance of ChatGPT in Cardiology and Vascular Pathologies
by: Hariri, Walid
Published: (2023)

A Thematic Framework for Analyzing Large-scale Self-reported Social Media Data on Opioid Use Disorder Treatment Using Buprenorphine Product
by: Basak, Madhusudan, et al.
Published: (2024)

Who Am I? History-Aware Profiles for Student Simulation in Tutoring Dialogues
by: Duan, Zhangqi, et al.
Published: (2026)

Who is better at math, Jenny or Jingzhen? Uncovering Stereotypes in Large Language Models
by: Siddique, Zara, et al.
Published: (2024)