Image de couverture de livre

Analyzing Dataset Annotation Quality Management in the Wild

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Klie, Jan-Christoph, de Castilho, Richard Eckart, Gurevych, Iryna
Format:	Preprint
Publié:	2023
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

On Efficient and Statistical Quality Estimation for Data Annotation
par: Klie, Jan-Christoph, et autres
Publié: (2024)

Is Peer Review Really in Decline? Analyzing Review Quality across Venues and Time
par: Kuznetsov, Ilia, et autres
Publié: (2026)

Citation Failure: Definition, Analysis and Efficient Mitigation
par: Buchmann, Jan, et autres
Publié: (2025)

SciCoQA: Quality Assurance for Scientific Paper--Code Alignment
par: Baumgärtner, Tim, et autres
Publié: (2026)

Re3: A Holistic Framework and Dataset for Modeling Collaborative Document Revision
par: Ruan, Qian, et autres
Publié: (2024)

Self-Rationalization in the Wild: A Large Scale Out-of-Distribution Evaluation on NLI-related tasks
par: Yang, Jing, et autres
Publié: (2025)

The Nature of NLP: Analyzing Contributions in NLP Papers
par: Pramanick, Aniket, et autres
Publié: (2024)

M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset
par: Geng, Jiahui, et autres
Publié: (2025)

Attribute or Abstain: Large Language Models as Long Document Assistants
par: Buchmann, Jan, et autres
Publié: (2024)

Like a Good Nearest Neighbor: Practical Content Moderation and Text Classification
par: Bates, Luke, et autres
Publié: (2023)

Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework
par: Dycke, Nils, et autres
Publié: (2025)

The Lou Dataset -- Exploring the Impact of Gender-Fair Language in German Text Classification
par: Waldis, Andreas, et autres
Publié: (2024)

PeerQA: A Scientific Question Answering Dataset from Peer Reviews
par: Baumgärtner, Tim, et autres
Publié: (2025)

ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety
par: Bates, Luke, et autres
Publié: (2025)

Diversity Over Size: On the Effect of Sample and Topic Sizes for Topic-Dependent Argument Mining Datasets
par: Schiller, Benjamin, et autres
Publié: (2022)

ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Links
par: Basch, Serwar, et autres
Publié: (2025)

SPARE: Single-Pass Annotation with Reference-Guided Evaluation for Automatic Process Supervision and Reward Modelling
par: Rizvi, Md Imbesat Hassan, et autres
Publié: (2025)

LazyReview A Dataset for Uncovering Lazy Thinking in NLP Peer Reviews
par: Purkayastha, Sukannya, et autres
Publié: (2025)

Commitment Checklist: Auditing Author Commitments in Peer Review
par: Chen, Chung-Chi, et autres
Publié: (2026)

Reviewing the Reviewer: Elevating Peer Review Quality through LLM-Guided Feedback
par: Purkayastha, Sukannya, et autres
Publié: (2026)

How to Handle Different Types of Out-of-Distribution Scenarios in Computational Argumentation? A Comprehensive and Fine-Grained Field Study
par: Waldis, Andreas, et autres
Publié: (2023)

CATfOOD: Counterfactual Augmented Training for Improving Out-of-Domain Performance and Calibration
par: Sachdeva, Rachneet, et autres
Publié: (2023)

How are Prompts Different in Terms of Sensitivity?
par: Lu, Sheng, et autres
Publié: (2023)

Identifying Aspects in Peer Reviews
par: Lu, Sheng, et autres
Publié: (2025)

Robust Utility-Preserving Text Anonymization Based on Large Language Models
par: Yang, Tianyu, et autres
Publié: (2024)

Token Weighting for Long-Range Language Modeling
par: Helm, Falko, et autres
Publié: (2025)

COVE: COntext and VEracity prediction for out-of-context images
par: Tonglet, Jonathan, et autres
Publié: (2025)

Dive into the Chasm: Probing the Gap between In- and Cross-Topic Generalization
par: Waldis, Andreas, et autres
Publié: (2024)

DARA: Decomposition-Alignment-Reasoning Autonomous Language Agent for Question Answering over Knowledge Graphs
par: Fang, Haishuo, et autres
Publié: (2024)

Turning Logic Against Itself : Probing Model Defenses Through Contrastive Questions
par: Sachdeva, Rachneet, et autres
Publié: (2025)

Overview of PerpectiveArg2024: The First Shared Task on Perspective Argument Retrieval
par: Falk, Neele, et autres
Publié: (2024)

LLM Roleplay: Simulating Human-Chatbot Interaction
par: Tamoyan, Hovhannes, et autres
Publié: (2024)

Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions
par: Ruan, Qian, et autres
Publié: (2024)

Expert Preference-based Evaluation of Automated Related Work Generation
par: Şahinuç, Furkan, et autres
Publié: (2025)

Reward Modeling for Scientific Writing Evaluation
par: Şahinuç, Furkan, et autres
Publié: (2026)

IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generators
par: Paul, Indraneil, et autres
Publié: (2024)

Is this chart lying to me? Automating the detection of misleading visualizations
par: Tonglet, Jonathan, et autres
Publié: (2025)

Document Structure in Long Document Transformers
par: Buchmann, Jan, et autres
Publié: (2024)

"Image, Tell me your story!" Predicting the original meta-context of visual misinformation
par: Tonglet, Jonathan, et autres
Publié: (2024)

The Inherent Limits of Pretrained LLMs: The Unexpected Convergence of Instruction Tuning and In-Context Learning Capabilities
par: Bigoulaeva, Irina, et autres
Publié: (2025)