Image de couverture de livre

Crowd-Calibrator: Can Annotator Disagreement Inform Calibration in Subjective Tasks?

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Khurana, Urja, Nalisnick, Eric, Fokkens, Antske, Swayamdipta, Swabha
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

DefVerify: Do Hate Speech Models Reflect Their Dataset's Definition?
par: Khurana, Urja, et autres
Publié: (2024)

Not All Subjectivity Is the Same! Defining Desiderata for the Evaluation of Subjectivity in NLP
par: Khurana, Urja, et autres
Publié: (2026)

The Role of Syntactic Span Preferences in Post-Hoc Explanation Disagreement
par: Kamp, Jonathan, et autres
Publié: (2024)

Annotating FrameNet via Structure-Conditioned Language Generation
par: Cui, Xinyue, et autres
Publié: (2024)

Investigating the Robustness of Modelling Decisions for Few-Shot Cross-Topic Stance Detection: A Preregistered Study
par: Reuver, Myrthe, et autres
Publié: (2024)

On the Low-Rank Parametrization of Reward Models for Controlled Language Generation
par: Troshin, Sergey, et autres
Publié: (2024)

Balancing the Scales: Reinforcement Learning for Fair Classification
par: Eshuijs, Leon, et autres
Publié: (2024)

Learning from Sufficient Rationales: Analysing the Relationship Between Explanation Faithfulness and Token-level Regularisation Strategies
par: Kamp, Jonathan, et autres
Publié: (2025)

Improving Causal Interventions in Amnesic Probing with Mean Projection or LEACE
par: Dobrzeniecka, Alicja, et autres
Publié: (2025)

Short-circuiting Shortcuts: Mechanistic Investigation of Shortcuts in Text Classification
par: Eshuijs, Leon, et autres
Publié: (2025)

Compare without Despair: Reliable Preference Evaluation with Generation Separability
par: Ghosh, Sayan, et autres
Publié: (2024)

Asking a Language Model for Diverse Responses
par: Troshin, Sergey, et autres
Publié: (2025)

When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks
par: Fleisig, Eve, et autres
Publié: (2023)

Understanding Dataset Difficulty with $\mathcal{V}$-Usable Information
par: Ethayarajh, Kawin, et autres
Publié: (2021)

How Reliable is Language Model Micro-Benchmarking?
par: Yauney, Gregory, et autres
Publié: (2025)

Evaluation Under Imperfect Benchmarks and Ratings: A Case Study in Text Simplification
par: Liu, Joseph, et autres
Publié: (2025)

Disentangling Geometry, Performance, and Training in Language Models
par: Kulkarni, Atharva, et autres
Publié: (2026)

Logits of API-Protected LLMs Leak Proprietary Information
par: Finlayson, Matthew, et autres
Publié: (2024)

BenchBrowser: Retrieving Evidence for Evaluating Benchmark Validity
par: Diddee, Harshita, et autres
Publié: (2026)

Labels have Human Values: Value Calibration of Subjective Tasks
par: Parappan, Mohammed Fayiz, et autres
Publié: (2026)

Sample, Align, Synthesize: Graph-Based Response Synthesis with ConGrs
par: Ghosh, Sayan, et autres
Publié: (2025)

Improving Language Model Personas via Rationalization with Psychological Scaffolds
par: Joshi, Brihi, et autres
Publié: (2025)

Better Language Model Inversion by Compactly Representing Next-Token Distributions
par: Nazir, Murtaza, et autres
Publié: (2025)

Teaching Models to Understand (but not Generate) High-risk Data
par: Wang, Ryan, et autres
Publié: (2025)

Task Calibration: Calibrating Large Language Models on Inference Tasks
par: Li, Yingjie, et autres
Publié: (2024)

NeuroComparatives: Neuro-Symbolic Distillation of Comparative Knowledge
par: Howard, Phillip, et autres
Publié: (2023)

Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge
par: Cui, Xinyue, et autres
Publié: (2025)

Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs
par: Troshin, Sergey, et autres
Publié: (2025)

Calibrating Probabilistic Object Detectors with Annotator Disagreement
par: Tan, Zhi Qin, et autres
Publié: (2026)

Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations
par: He, Keyu, et autres
Publié: (2025)

Crowded in B-Space: Calibrating Shared Directions for LoRA Merging
par: Tang, Yixuan, et autres
Publié: (2026)

Leveraging Annotator Disagreement for Text Classification
par: Xu, Jin, et autres
Publié: (2024)

Through the Lens of Split Vote: Exploring Disagreement, Difficulty and Calibration in Legal Case Outcome Classification
par: Xu, Shanshan, et autres
Publié: (2024)

Generative Explanations for Program Synthesizers
par: Nazari, Amirmohammad, et autres
Publié: (2024)

Active Learning for Multilingual Fingerspelling Corpora
par: Wang, Shuai, et autres
Publié: (2023)

Can Reasoning Help Large Language Models Capture Human Annotator Disagreement?
par: Ni, Jingwei, et autres
Publié: (2025)

JuniperLiu at CoMeDi Shared Task: Models as Annotators in Lexical Semantics Disagreements
par: Liu, Zhu, et autres
Publié: (2024)

OATH-Frames: Characterizing Online Attitudes Towards Homelessness with LLM Assistants
par: Ranjit, Jaspreet, et autres
Publié: (2024)

Aligning LLM Uncertainty with Human Disagreement in Subjectivity Analysis
par: Lu, Junyu, et autres
Publié: (2026)

Funzac at CoMeDi Shared Task: Modeling Annotator Disagreement from Word-In-Context Perspectives
par: Sarumi, Olufunke O., et autres
Publié: (2025)