Cover Image

Do We Still Need Humans in the Loop? Comparing Human and LLM Annotation in Active Learning for Hostility Detection

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hakimi, Ahmad Dawar, Hirlimann, Lea, Augenstein, Isabelle, Schütze, Hinrich
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

From Human Annotation to Automation: LLM-in-the-Loop Active Learning for Arabic Sentiment Analysis
by: Refai, Dania, et al.
Published: (2025)

MemeScouts@LT-EDI 2026: Asking the Right Questions -- Prompted Weak Supervision for Meme Hate Speech Detection
by: Bueno, Ivo, et al.
Published: (2026)

Time Course MechInterp: Analyzing the Evolution of Components and Knowledge in Large Language Models
by: Hakimi, Ahmad Dawar, et al.
Published: (2025)

On Relation-Specific Neurons in Large Language Models
by: Liu, Yihong, et al.
Published: (2025)

SLAyiNG: Towards Queer Language Processing
by: Veloso, Leonor, et al.
Published: (2025)

Relational Linearity is a Predictor of Hallucinations
by: Lu, Yuetian, et al.
Published: (2026)

Exploring the Role of Transliteration in In-Context Learning for Low-resource Languages Written in Non-Latin Scripts
by: Ma, Chunlan, et al.
Published: (2024)

A Federated Approach to Few-Shot Hate Speech Detection for Marginalized Communities
by: Ye, Haotian, et al.
Published: (2024)

GlotCC: An Open Broad-Coverage CommonCrawl Corpus and Pipeline for Minority Languages
by: Kargaran, Amir Hossein, et al.
Published: (2024)

Comparing LLM Text Annotation Skills: A Study on Human Rights Violations in Social Media Data
by: Nemkova, Poli Apollinaire, et al.
Published: (2025)

Measuring and Benchmarking Large Language Models' Capabilities to Generate Persuasive Language
by: Pauli, Amalie Brogaard, et al.
Published: (2024)

MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer
by: Ye, Haotian, et al.
Published: (2024)

Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation
by: Rouzegar, Hamidreza, et al.
Published: (2024)

Analysing Differences in Persuasive Language in LLM-Generated Text: Uncovering Stereotypical Gender Patterns
by: Pauli, Amalie Brogaard, et al.
Published: (2026)

Presumed Cultural Identity: How Names Shape LLM Responses
by: Pawar, Siddhesh, et al.
Published: (2025)

Do Benchmarks Underestimate LLM Performance? Evaluating Hallucination Detection With LLM-First Human-Adjudicated Assessment
by: Atasoy, I. F., et al.
Published: (2026)

Augmenting Human Evaluation with LLM Judges: How Many Human Reviews Do You Need?
by: Kim, Jane Paik
Published: (2026)

Can Community Notes Replace Professional Fact-Checkers?
by: Borenstein, Nadav, et al.
Published: (2025)

ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge
by: Taghavi, Zeinab Sadat, et al.
Published: (2025)

BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
by: Nie, Ercong, et al.
Published: (2024)

Can We Still Hear the Accent? Investigating the Resilience of Native Language Signals in the LLM Era
by: Utami, Nabelanita, et al.
Published: (2026)

Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checking
by: Warren, Greta, et al.
Published: (2025)

Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?
by: Lu, Yi-Long, et al.
Published: (2025)

Stress Testing Factual Consistency Metrics for Long-Document Summarization
by: Mujahid, Zain Muhammad, et al.
Published: (2025)

Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement
by: Islam, Sekh Mainul, et al.
Published: (2025)

GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network
by: Yuan, Shuzhou, et al.
Published: (2024)

Enhancing Robustness of Autoregressive Language Models against Orthographic Attacks via Pixel-based Approach
by: Yang, Han, et al.
Published: (2025)

Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena
by: Weissweiler, Leonie, et al.
Published: (2024)

LongForm: Effective Instruction Tuning with Reverse Instructions
by: Köksal, Abdullatif, et al.
Published: (2023)

CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation
by: Ziegler, Ingo, et al.
Published: (2024)

Semantic Sensitivities and Inconsistent Predictions: Measuring the Fragility of NLI Models
by: Arakelyan, Erik, et al.
Published: (2024)

Text Annotation via Inductive Coding: Comparing Human Experts to LLMs in Qualitative Data Analysis
by: Parfenova, Angelina, et al.
Published: (2025)

Claim Verification in the Age of Large Language Models: A Survey
by: Dmonte, Alphaeus, et al.
Published: (2024)

Do We Need Frontier Models to Verify Mathematical Proofs?
by: Naik, Aaditya, et al.
Published: (2026)

Why Do We Laugh? Annotation and Taxonomy Generation for Laughable Contexts in Spontaneous Text Conversation
by: Inoue, Koji, et al.
Published: (2025)

SynDARin: Synthesising Datasets for Automated Reasoning in Low-Resource Languages
by: Ghazaryan, Gayane, et al.
Published: (2024)

How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations
by: Wang, Zora Zhiruo, et al.
Published: (2025)

Revealing the Parametric Knowledge of Language Models: A Unified Framework for Attribution Methods
by: Yu, Haeun, et al.
Published: (2024)

Large Language Models Are Effective Human Annotation Assistants, But Not Good Independent Annotators
by: Gu, Feng, et al.
Published: (2025)

Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targets
by: Giorgi, Tommaso, et al.
Published: (2024)