Cover Image

The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detection

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Horych, Tomas, Mandl, Christoph, Ruas, Terry, Greiner-Petter, Andre, Gipp, Bela, Aizawa, Akiko, Spinde, Timo
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MAGPIE: Multi-Task Media-Bias Analysis Generalization for Pre-Trained Identification of Expressions
by: Horych, Tomáš, et al.
Published: (2024)

The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Bias
by: Spinde, Timo, et al.
Published: (2023)

Overview of the Plagiarism Detection Task at PAN 2025
by: Greiner-Petter, André, et al.
Published: (2025)

Automated Detection of Media Bias
by: Spinde, Timo
Published: (2025)

What's Wrong? Refining Meeting Summaries with LLM Feedback
by: Kirstein, Frederic, et al.
Published: (2024)

Is my Meeting Summary Good? Estimating Quality with a Multi-LLM Evaluator
by: Kirstein, Frederic, et al.
Published: (2024)

Can LLMs Master Math? Investigating Large Language Models on Math Stack Exchange
by: Satpute, Ankit, et al.
Published: (2024)

Aspect-Aware Content-Based Recommendations for Mathematical Research Papers
by: Satpute, Ankit, et al.
Published: (2026)

Paraphrase Types for Generation and Detection
by: Wahle, Jan Philip, et al.
Published: (2023)

Piecing Together Cross-Document Coreference Resolution Datasets: Systematic Dataset Analysis and Unification
by: Zhukova, Anastasia, et al.
Published: (2026)

Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting Summarization
by: Kirstein, Frederic, et al.
Published: (2024)

Taxonomy of Mathematical Plagiarism
by: Satpute, Ankit, et al.
Published: (2024)

Enhancing Media Literacy: The Effectiveness of (Human) Annotations and Bias Visualizations on Bias Detection
by: Spinde, Timo, et al.
Published: (2024)

What's in the News? Towards Identification of Bias by Commission, Omission, and Source Selection (COSS)
by: Zhukova, Anastasia, et al.
Published: (2025)

Big Tech-Funded AI Papers Have Higher Citation Impact, Greater Insularity, and Larger Recency Bias
by: Gnewuch, Max Martin, et al.
Published: (2025)

Through the LLM Looking Glass: A Socratic Probing of Donkeys, Elephants, and Markets
by: Kennedy, Molly, et al.
Published: (2025)

D3: A Massive Dataset of Scholarly Metadata for Analyzing the State of Computer Science Research
by: Wahle, Jan Philip, et al.
Published: (2022)

Re-FRAME the Meeting Summarization SCOPE: Fact-Based Summarization and Personalization via Questions
by: Kirstein, Frederic, et al.
Published: (2025)

Citation Amnesia: On The Recency Bias of NLP and Other Academic Fields
by: Wahle, Jan Philip, et al.
Published: (2024)

Who Watches the Watchmen? Humans Disagree With Translation Metrics on Unseen Domains
by: Schmidt, Finn, et al.
Published: (2026)

What's under the hood: Investigating Automatic Metrics on Meeting Summarization
by: Kirstein, Frederic, et al.
Published: (2024)

Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges
by: Becker, Jonas, et al.
Published: (2024)

Paraphrase Types Elicit Prompt Engineering Capabilities
by: Wahle, Jan Philip, et al.
Published: (2024)

Towards Human Understanding of Paraphrase Types in Large Language Models
by: Meier, Dominik, et al.
Published: (2024)

CADS: A Systematic Literature Review on the Challenges of Abstractive Dialogue Summarization
by: Kirstein, Frederic, et al.
Published: (2024)

How Large Language Models are Transforming Machine-Paraphrased Plagiarism
by: Wahle, Jan Philip, et al.
Published: (2022)

CiteAssist: A System for Automated Preprint Citation and BibTeX Generation
by: Kaesberg, Lars Benedikt, et al.
Published: (2024)

SPaRC: A Spatial Pathfinding Reasoning Challenge
by: Kaesberg, Lars Benedikt, et al.
Published: (2025)

Leveraging Large Language Models for Automated Definition Extraction with TaxoMatic A Case Study on Media Bias
by: Spinde, Timo, et al.
Published: (2025)

DefExtra
by: Kučera, Filip, et al.
Published: (2026)

SciDef: Automating Definition Extraction from Academic Literature with Large Language Models
by: Kučera, Filip, et al.
Published: (2026)

Testing the Generalization of Neural Language Models for COVID-19 Misinformation Detection
by: Wahle, Jan Philip, et al.
Published: (2021)

You need to MIMIC to get FAME: Solving Meeting Transcript Scarcity with a Multi-Agent Conversations
by: Kirstein, Frederic, et al.
Published: (2025)

TrojanStego: Your Language Model Can Secretly Be A Steganographic Privacy Leaking Agent
by: Meier, Dominik, et al.
Published: (2025)

Beyond a Single Direction: Chain-of-Thought Disrupts Simple Steering of Refusal
by: Yang, Kia-Jüng, et al.
Published: (2026)

We are Who We Cite: Bridges of Influence Between Natural Language Processing and Other Academic Fields
by: Wahle, Jan Philip, et al.
Published: (2023)

Voting or Consensus? Decision-Making in Multi-Agent Debate
by: Kaesberg, Lars Benedikt, et al.
Published: (2025)

Encoded but Not Routed: Explaining the Table-Chart Gap in Scientific Claim Verification
by: Kumar, Sunisth, et al.
Published: (2026)

News Ninja: Gamified Annotation of Linguistic Bias in Online News
by: Hinterreiter, Smi, et al.
Published: (2024)

Refining and Reusing Annotation Guidelines for LLM Annotation
by: Kim, Kon Woo, et al.
Published: (2026)