Cover Image

Human Label Variation as Stable Signal: Learning Annotator-Specific Explanation Behavior via Cross-Annotator Preference Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Beiduo, Hong, Pingjun, Zhang, Ziyun, Roth, Benjamin, Korhonen, Anna, Plank, Barbara
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanations
by: Hong, Pingjun, et al.
Published: (2025)

A Rose by Any Other Name: LLM-Generated Explanations Are Good Proxies for Human Explanations to Collect Label Distributions on NLI
by: Chen, Beiduo, et al.
Published: (2024)

Threading the Needle: Reweaving Chain-of-Thought Reasoning to Explain Human Label Variation
by: Chen, Beiduo, et al.
Published: (2025)

LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference
by: Hong, Pingjun, et al.
Published: (2025)

Decoupling the Effect of Chain-of-Thought Reasoning: A Human Label Variation Perspective
by: Chen, Beiduo, et al.
Published: (2026)

Do LLM Self-Explanations Help Users Predict Model Behavior? Evaluating Counterfactual Simulatability with Pragmatic Perturbations
by: Hong, Pingjun, et al.
Published: (2026)

"Seeing the Big through the Small": Can LLMs Approximate Human Judgment Distributions on NLI from a Few Explanations?
by: Chen, Beiduo, et al.
Published: (2024)

Not All Explanations Simulate Equally: Comparing Verbalized Feature Attributions and Self-Generated Rationales
by: Hong, Pingjun, et al.
Published: (2026)

Different Tastes of Entities: Investigating Human Label Variation in Named Entity Annotations
by: Peng, Siyao, et al.
Published: (2024)

VariErr NLI: Separating Annotation Error from Human Label Variation
by: Weber-Genzel, Leon, et al.
Published: (2024)

Evaluating Large Language Models for Cross-Lingual Retrieval
by: Zuo, Longfei, et al.
Published: (2025)

From Noise to Signal to Selbstzweck: Reframing Human Label Variation in the Era of Post-training in NLP
by: Xu, Shanshan, et al.
Published: (2025)

MaiBaam Annotation Guidelines
by: Blaschke, Verena, et al.
Published: (2024)

Reasoning that Travels: Dissecting How Chain-of-Thought Transfers Across Models
by: Cheng, Xinyuan, et al.
Published: (2026)

Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?
by: Baan, Joris, et al.
Published: (2024)

MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMs
by: Zhao, Raoyuan, et al.
Published: (2025)

Revisiting Active Learning under (Human) Label Variation
by: Gruber, Cornelia, et al.
Published: (2025)

The Ecological Fallacy in Annotation: Modelling Human Label Variation goes beyond Sociodemographics
by: Orlikowski, Matthias, et al.
Published: (2023)

Disagreeing Rationales: Rethinking Classification and Explainability Evaluation in Hate Speech Detection
by: Muscato, Benedetta, et al.
Published: (2026)

Tokenization Preference for Human and Machine Learning Model: An Annotation Study
by: Hiraoka, Tatsuya, et al.
Published: (2023)

Fine-Grained Perspectives: Modeling Explanations with Annotator-Specific Rationales
by: Sarumi, Olufunke O., et al.
Published: (2026)

When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training
by: Körner, Felicia, et al.
Published: (2026)

EEVEE: An Easy Annotation Tool for Natural Language Processing
by: Sorensen, Axel, et al.
Published: (2024)

Mitigating Cross-Lingual Cultural Inconsistencies in LLMs via Consensus-Driven Preference Optimisation
by: Resck, Lucas, et al.
Published: (2026)

Donkii: Can Annotation Error Detection Methods Find Errors in Instruction-Tuning Datasets?
by: Weber-Genzel, Leon, et al.
Published: (2023)

Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement
by: Abercrombie, Gavin, et al.
Published: (2023)

Large Language Models Are Effective Human Annotation Assistants, But Not Good Independent Annotators
by: Gu, Feng, et al.
Published: (2025)

ZEBRA: Leveraging Model-Behavioral Knowledge for Zero-Annotation Preference Dataset Construction
by: Jung, Jeesu, et al.
Published: (2025)

BoN Appetit Team at LeWiDi-2025: Best-of-N Test-time Scaling Can Not Stomach Annotation Disagreements (Yet)
by: Ruiz, Tomas, et al.
Published: (2025)

Metric-Dependent Annotation Saturation for Learning from Label Distributions
by: Kohli, Guneet
Published: (2026)

EVADE: LLM-Based Explanation Generation and Validation for Error Detection in NLI
by: Zuo, Longfei, et al.
Published: (2025)

Crossing Domains without Labels: Distant Supervision for Term Extraction
by: Senger, Elena, et al.
Published: (2025)

Users as Annotators: LLM Preference Learning from Comparison Mode
by: Cai, Zhongze, et al.
Published: (2025)

Who and What? Using Linguistic Features and Annotator Characteristics to Analyze Annotation Variation
by: Maurer, Maximilian, et al.
Published: (2026)

To Err Is Human; To Annotate, SILICON? Toward Robust Reproducibility in LLM Annotation
by: Cheng, Xiang, et al.
Published: (2024)

Not All Preference Pairs Are Created Equal: A Recipe for Annotation-Efficient Iterative Preference Learning
by: Yang, Sen, et al.
Published: (2024)

From Random to Informed Data Selection: A Diversity-Based Approach to Optimize Human Annotation and Few-Shot Learning
by: Alcoforado, Alexandre, et al.
Published: (2024)

Hybrid Annotation for Propaganda Detection: Integrating LLM Pre-Annotations with Human Intelligence
by: Sahitaj, Ariana, et al.
Published: (2025)

Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
by: Mondorf, Philipp, et al.
Published: (2024)

HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages
by: Wang, Zhilin, et al.
Published: (2025)