Cover Image

The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fleisig, Eve, Blodgett, Su Lin, Klein, Dan, Talat, Zeerak
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language Computers and Society
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Capabilities Approach to Studying Bias and Harm in Language Technologies
by: Nigatu, Hellina Hailu, et al.
Published: (2024)

Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination
by: Fleisig, Eve, et al.
Published: (2024)

Understanding "Democratization" in NLP and ML Research
by: Subramonian, Arjun, et al.
Published: (2024)

Subjective $\textit{Isms}$? On the Danger of Conflating Hate and Offence in Abusive Language Detection
by: Curry, Amanda Cercas, et al.
Published: (2024)

Impoverished Language Technology: The Lack of (Social) Class in NLP
by: Curry, Amanda Cercas, et al.
Published: (2024)

FedMental: Evaluating Federated Learning for Mental Health Detection from Social Media Data
by: Abdelkadir, Nuredin Ali, et al.
Published: (2026)

Balancing Quality and Variation: Spam Filtering Distorts Data Label Distributions
by: Fleisig, Eve, et al.
Published: (2025)

When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks
by: Fleisig, Eve, et al.
Published: (2023)

MLLM-as-a-Judge for Image Safety without Human Labeling
by: Wang, Zhenting, et al.
Published: (2024)

Exploitation All the Way Down: Calling out the Root Cause of Bad Online Experiences for Users of the "Majority World"
by: Nigatu, Hellina Hailu, et al.
Published: (2024)

PluriHarms: Benchmarking the Full Spectrum of Human Judgments on AI Harm
by: Li, Jing-Jing, et al.
Published: (2026)

Augmenting Human-Annotated Training Data with Large Language Model Generation and Distillation in Open-Response Assessment
by: Borchers, Conrad, et al.
Published: (2025)

Protected group bias and stereotypes in Large Language Models
by: Kotek, Hadas, et al.
Published: (2024)

Position: the Stochastic Parrot in the Coal Mine. Model Collapse is a Threat to Low-Resource Communities
by: Jarvis, Devon, et al.
Published: (2026)

Questionnaire Responses Do not Capture the Safety of AI Agents
by: Hellrigel-Holderbaum, Max, et al.
Published: (2026)

Ghostbuster: Detecting Text Ghostwritten by Large Language Models
by: Verma, Vivek, et al.
Published: (2023)

LLM Analysis of 150+ years of German Parliamentary Debates on Migration Reveals Shift from Post-War Solidarity to Anti-Solidarity in the Last Decade
by: Kostikova, Aida, et al.
Published: (2025)

The Dark Side of ChatGPT: Legal and Ethical Challenges from Stochastic Parrots and Hallucination
by: Li, Zihao
Published: (2023)

Operationalizing Automated Essay Scoring: A Human-Aware Approach
by: Plasencia-Calaña, Yenisel
Published: (2025)

Mapping Social Choice Theory to RLHF
by: Dai, Jessica, et al.
Published: (2024)

Hypothesis Testing for Quantifying LLM-Human Misalignment in Multiple Choice Settings
by: Hong, Harbin, et al.
Published: (2025)

Diagnosing Hate Speech Classification: Where Do Humans and Machines Disagree, and Why?
by: Yang, Xilin
Published: (2024)

Vicarious Offense and Noise Audit of Offensive Speech Classifiers: Unifying Human and Machine Disagreement on What is Offensive
by: Weerasooriya, Tharindu Cyril, et al.
Published: (2023)

Challenging Assumptions in Learning Generic Text Style Embeddings
by: Ostheimer, Phil, et al.
Published: (2025)

KPoEM: A Human-Annotated Dataset for Emotion Classification and RAG-Based Poetry Generation in Korean Modern Poetry
by: Lim, Iro, et al.
Published: (2025)

Simulated Adoption: Decoupling Magnitude and Direction in LLM In-Context Conflict Resolution
by: Zhang, Long, et al.
Published: (2026)

Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models
by: Ungless, Eddie L., et al.
Published: (2024)

The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations
by: Zhang, Long, et al.
Published: (2026)

Online Learning Defense against Iterative Jailbreak Attacks via Prompt Optimization
by: Kaneko, Masahiro, et al.
Published: (2025)

Big AI's Regulatory Capture: Mapping Industry Interference and Government Complicity
by: Birhane, Abeba, et al.
Published: (2026)

Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications
by: Dong, Wenhan, et al.
Published: (2025)

Foundational Challenges in Assuring Alignment and Safety of Large Language Models
by: Anwar, Usman, et al.
Published: (2024)

Embracing Imperfection: Simulating Students with Diverse Cognitive Levels Using LLM-based Agents
by: Wu, Tao, et al.
Published: (2025)

"I Am the One and Only, Your Cyber BFF": Understanding the Impact of GenAI Requires Understanding the Impact of Anthropomorphic AI
by: Cheng, Myra, et al.
Published: (2024)

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
by: Harvey, Emma, et al.
Published: (2025)

ELMES: An Automated Framework for Evaluating Large Language Models in Educational Scenarios
by: Wei, Shou'ang, et al.
Published: (2025)

LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet
by: Li, Nathaniel, et al.
Published: (2024)

PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents
by: Yang, Qisen, et al.
Published: (2024)

AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy
by: Schoenegger, Philipp, et al.
Published: (2024)

Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments
by: Zhou, Han, et al.
Published: (2024)