Cover Image

Metric-Dependent Annotation Saturation for Learning from Label Distributions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Kohli, Guneet
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Aggregating Soft Labels from Crowd Annotations Improves Uncertainty Estimation Under Distribution Shift
by: Wright, Dustin, et al.
Published: (2022)

Human Label Variation as Stable Signal: Learning Annotator-Specific Explanation Behavior via Cross-Annotator Preference Optimization
by: Chen, Beiduo, et al.
Published: (2026)

SciAnnotate: A Tool for Integrating Weak Labeling Sources for Sequence Labeling
by: Liu, Mengyang, et al.
Published: (2022)

Dependency Annotation of Ottoman Turkish with Multilingual BERT
by: Özateş, Şaziye Betül, et al.
Published: (2024)

Dependency Graph Parsing as Sequence Labeling
by: Ezquerro, Ana, et al.
Published: (2024)

VariErr NLI: Separating Annotation Error from Human Label Variation
by: Weber-Genzel, Leon, et al.
Published: (2024)

Textless Dependency Parsing by Labeled Sequence Prediction
by: Kando, Shunsuke, et al.
Published: (2024)

UCxn: Typologically Informed Annotation of Constructions Atop Universal Dependencies
by: Weissweiler, Leonie, et al.
Published: (2024)

Learning from Saturated Data: Signals Beyond Correctness for LLM Training
by: Hiss, Hanno, et al.
Published: (2026)

Scaling Public Health Text Annotation: Zero-Shot Learning vs. Crowdsourcing for Improved Efficiency and Labeling Accuracy
by: Kazari, Kamyar, et al.
Published: (2025)

The Ecological Fallacy in Annotation: Modelling Human Label Variation goes beyond Sociodemographics
by: Orlikowski, Matthias, et al.
Published: (2023)

Different Tastes of Entities: Investigating Human Label Variation in Named Entity Annotations
by: Peng, Siyao, et al.
Published: (2024)

Counting on Consensus: Selecting the Right Inter-annotator Agreement Metric for NLP Annotation and Evaluation
by: James, Joseph
Published: (2026)

Evaluating the IWSLT2023 Speech Translation Tasks: Human Annotations, Automatic Metrics, and Segmentation
by: Sperber, Matthias, et al.
Published: (2024)

L3Ms -- Lagrange Large Language Models
by: Dhillon, Guneet S., et al.
Published: (2024)

GroundCocoa: A Benchmark for Evaluating Compositional & Conditional Reasoning in Language Models
by: Kohli, Harsh, et al.
Published: (2024)

A Measure of the System Dependence of Automated Metrics
by: von Däniken, Pius, et al.
Published: (2024)

Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement
by: Abercrombie, Gavin, et al.
Published: (2023)

A Positive-Unlabeled Metric Learning Framework for Document-Level Relation Extraction with Incomplete Labeling
by: Wang, Ye, et al.
Published: (2023)

A Comparative Study on Annotation Quality of Crowdsourcing and LLM via Label Aggregation
by: Li, Jiyi
Published: (2024)

The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detection
by: Horych, Tomas, et al.
Published: (2024)

Label Distribution Learning-Enhanced Dual-KNN for Text Classification
by: Yuan, Bo, et al.
Published: (2025)

EvoPool: Evolutionary Programmatic Annotation for Label-Efficient Specialized Supervision
by: Xu, Tianyi, et al.
Published: (2026)

In-Context Example Ordering Guided by Label Distributions
by: Xu, Zhichao, et al.
Published: (2024)

Revisiting Semantic Role Labeling: Efficient Structured Inference with Dependency-Informed Analysis
by: Youm, Sangpil, et al.
Published: (2026)

Metadata Predictability Is Not Evidence Dependence: An Intervention-Based Audit for Weak-Label Benchmarks
by: Shao, Kan
Published: (2026)

Label Dependencies-aware Set Prediction Networks for Multi-label Text Classification
by: Xinkai, Du, et al.
Published: (2023)

When Annotators Agree but Labels Disagree: The Projection Problem in Stance Detection
by: Zhang, Bowen
Published: (2026)

Humans Hallucinate Too: Language Models Identify and Correct Subjective Annotation Errors With Label-in-a-Haystack Prompts
by: Chochlakis, Georgios, et al.
Published: (2025)

A Fully Automated Pipeline for Conversational Discourse Annotation: Tree Scheme Generation and Labeling with Large Language Models
by: Petukhova, Kseniia, et al.
Published: (2025)

Learning Who Disagrees: Demographic Importance Weighting for Modeling Annotator Distributions with DiADEM
by: Shetty, Samay U., et al.
Published: (2026)

The Ouroboros of Benchmarking: Reasoning Evaluation in an Era of Saturation
by: Deveci, İbrahim Ethem, et al.
Published: (2025)

From Annotation to Adaptation: Metrics, Synthetic Data, and Aspect Extraction for Aspect-Based Sentiment Analysis with Large Language Models
by: Neveditsin, Nikita, et al.
Published: (2025)

What's Mine becomes Yours: Defining, Annotating and Detecting Context-Dependent Paraphrases in News Interview Dialogs
by: Wegmann, Anna, et al.
Published: (2024)

Bias in LLMs as Annotators: The Effect of Party Cues on Labelling Decision by Large Language Models
by: Vera, Sebastian Vallejo, et al.
Published: (2024)

Knowledge Distillation in Automated Annotation: Supervised Text Classification with LLM-Generated Training Labels
by: Pangakis, Nicholas, et al.
Published: (2024)

Learning Nested Named Entity Recognition from Flat Annotations
by: Rozhkov, Igor, et al.
Published: (2026)

Intuitionistic Fuzzy Sets for Large Language Model Data Annotation: A Novel Approach to Side-by-Side Preference Labeling
by: Du, Yimin
Published: (2025)

In-Context Learning with Noisy Labels
by: Kang, Junyong, et al.
Published: (2024)

EM2LDL: A Multilingual Speech Corpus for Mixed Emotion Recognition through Label Distribution Learning
by: Li, Xingfeng, et al.
Published: (2025)