Cover Image

Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Tasks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Schroeder, Hope, Roy, Deb, Kabbara, Jad
Format:	Preprint
Published:	2025
Subjects:	Computers and Society
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

An AI-Powered Framework for Analyzing Collective Idea Evolution in Deliberative Assemblies
by: Poole-Dayan, Elinor, et al.
Published: (2025)

Computational Analysis of Conversation Dynamics through Participant Responsivity
by: Hughes, Margaret, et al.
Published: (2025)

LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users
by: Poole-Dayan, Elinor, et al.
Published: (2024)

PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits
by: Jiang, Hang, et al.
Published: (2023)

Bridging Context Gaps: Enhancing Comprehension in Long-Form Social Conversations Through Contextualized Excerpts
by: Mohanty, Shrestha, et al.
Published: (2024)

ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddings
by: Brannon, William, et al.
Published: (2023)

Data Authenticity, Consent, & Provenance for AI are all broken: what will it take to fix them?
by: Longpre, Shayne, et al.
Published: (2024)

Conversation Networks
by: Roy, Deb, et al.
Published: (2025)

Human-in-the-Loop LLM Grading for Handwritten Mathematics Assessments
by: Vanhoyweghen, Arne, et al.
Published: (2026)

'Put the Car on the Stand': SMT-based Oracles for Investigating Decisions
by: Judson, Samuel, et al.
Published: (2023)

Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review
by: Pang, Rock Yuren, et al.
Published: (2025)

The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks
by: Møller, Anders Giovanni, et al.
Published: (2023)

Discretion in the Loop: Human Expertise in Algorithm-Assisted College Advising
by: Schechtman, Kara, et al.
Published: (2025)

Consensus and Subjectivity of Skin Tone Annotation for ML Fairness
by: Schumann, Candice, et al.
Published: (2023)

Annotator in the Loop: A Case Study of In-Depth Rater Engagement to Create a Bridging Benchmark Dataset
by: Schmer-Galunder, Sonja, et al.
Published: (2024)

On the Relationship between Truth and Political Bias in Language Models
by: Fulay, Suyash, et al.
Published: (2024)

The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation
by: Munir, Sheza, et al.
Published: (2026)

Investigating Bias in LLM-Based Bias Detection: Disparities between LLMs and Human Perception
by: Lin, Luyang, et al.
Published: (2024)

An Annotated Reading of 'The Singer of Tales' in the LLM Era
by: Varshney, Kush R.
Published: (2025)

Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models
by: Kumar, Abhishek, et al.
Published: (2024)

No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy
by: Wilson, Kyra, et al.
Published: (2025)

Culturally Adaptive Explainable LLM Assessment for Multilingual Information Disorder: A Human-in-the-Loop Approach
by: Jouneghani, Maziar Kianimoghadam
Published: (2026)

Don't Blame the Data, Blame the Model: Understanding Noise and Bias When Learning from Subjective Annotations
by: Anand, Abhishek, et al.
Published: (2024)

Putting green software principles into practice
by: Uther, James
Published: (2026)

Sycophancy Claims about Language Models: The Missing Human-in-the-Loop
by: Batzner, Jan, et al.
Published: (2025)

Putting GenAI on Notice: GenAI Exceptionalism and Contract Law
by: Atkinson, David
Published: (2025)

Smart Metadata in Action: The Social Impact Data Commons
by: Schroeder, Joanna, et al.
Published: (2025)

AI for Just Work: Constructing Diverse Imaginations of AI beyond "Replacing Humans"
by: Jin, Weina, et al.
Published: (2025)

DART: Mitigating Harm Drift in Difference-Aware LLMs via Distill-Audit-Repair Training
by: Pan, Ziwen, et al.
Published: (2026)

Designing Effective LLM-Assisted Interfaces for Curriculum Development
by: Faraji, Abdolali, et al.
Published: (2025)

"Flipped" University: LLM-Assisted Lifelong Learning Environment
by: Krinkin, Kirill, et al.
Published: (2024)

Personalized Parsons Puzzles as Scaffolding Enhance Practice Engagement Over Just Showing LLM-Powered Solutions
by: Hou, Xinying, et al.
Published: (2025)

A Perspective on Crowdsourcing and Human-in-the-Loop Workflows in Precision Health
by: Washington, Peter
Published: (2023)

Farewell to Westphalia: Crypto Sovereignty and Post-Nation-State Governaance
by: Hope, Jarrad, et al.
Published: (2025)

GPT is Not an Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction
by: Felkner, Virginia K., et al.
Published: (2024)

Should There be a Teacher In-the-Loop? A Study of Generative AI Personalized Tasks Middle School
by: Walkington, Candace, et al.
Published: (2026)

IDEAlign: Comparing Large Language Models to Human Experts in Open-ended Interpretive Annotations
by: Nam, Hyunji, et al.
Published: (2025)

Dependency-Aware Task Offloading in Multi-UAV Assisted Collaborative Mobile Edge Computing
by: Zhao, Zhenyu, et al.
Published: (2025)

FairSCOSCA: Fairness At Arterial Signals -- Just Around The Corner
by: Riehl, Kevin, et al.
Published: (2026)

LLM-Assisted Replication for Quantitative Social Science
by: Kubota, So, et al.
Published: (2026)