Copertina

Estimating Agreement by Chance for Sequence Annotation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Diya, Rosé, Carolyn, Yuan, Ao, Zhou, Chunxiao
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Distilling Multi-Scale Knowledge for Event Temporal Relation Extraction
di: Yao, Hao-Ren, et al.
Pubblicazione: (2022)

Counting on Consensus: Selecting the Right Inter-annotator Agreement Metric for NLP Annotation and Evaluation
di: James, Joseph
Pubblicazione: (2026)

Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement
di: Abercrombie, Gavin, et al.
Pubblicazione: (2023)

Structured Disagreement in Health-Literacy Annotation: Epistemic Stability, Conceptual Difficulty, and Agreement-Stratified Inference
di: Kellert, Olga, et al.
Pubblicazione: (2026)

Beyond Majority Voting: Agreement-Based Clustering to Model Annotator Perspectives in Subjective NLP Tasks
di: Belay, Tadesse Destaw, et al.
Pubblicazione: (2026)

On Code-Induced Reasoning in LLMs
di: Waheed, Abdul, et al.
Pubblicazione: (2025)

Cognitive Agent Compilation for Explicit Problem Solver Modeling
di: Moon, Hyeongdon, et al.
Pubblicazione: (2026)

SciAnnotate: A Tool for Integrating Weak Labeling Sources for Sequence Labeling
di: Liu, Mengyang, et al.
Pubblicazione: (2022)

GPTs Are Multilingual Annotators for Sequence Generation Tasks
di: Choi, Juhwan, et al.
Pubblicazione: (2024)

Reflective Agreement: Combining Self-Mixture of Agents with a Sequence Tagger for Robust Event Extraction
di: Haji, Fatemeh, et al.
Pubblicazione: (2025)

Automated Clinical Data Extraction with Knowledge Conditioned LLMs
di: Li, Diya, et al.
Pubblicazione: (2024)

ErAConD : Error Annotated Conversational Dialog Dataset for Grammatical Error Correction
di: Yuan, Xun, et al.
Pubblicazione: (2021)

Estimating the Level of Dialectness Predicts Interannotator Agreement in Multi-dialect Arabic Datasets
di: Keleg, Amr, et al.
Pubblicazione: (2024)

Large Language Models as Annotators for Machine Translation Quality Estimation
di: Wang, Sidi, et al.
Pubblicazione: (2026)

BiST: A Gold Standard Bangla-English Bilingual Corpus for Sentence Structure and Tense Classification with Inter-Annotator Agreement
di: Shafi, Abdullah Al, et al.
Pubblicazione: (2026)

APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention
di: Huang, Yuxiang, et al.
Pubblicazione: (2026)

ReaComp: Compiling LLM Reasoning into Symbolic Solvers for Efficient Program Synthesis
di: Naik, Atharva, et al.
Pubblicazione: (2026)

ChartEditBench: Evaluating Grounded Multi-Turn Chart Editing in Multimodal Language Models
di: Kapadnis, Manav Nitin, et al.
Pubblicazione: (2026)

Argument Reconstruction as Supervision for Critical Thinking in LLMs
di: Ryu, Hyun, et al.
Pubblicazione: (2026)

CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smells
di: Naik, Atharva, et al.
Pubblicazione: (2024)

Data Augmentation for Code Translation with Comparable Corpora and Multiple References
di: Xie, Yiqing, et al.
Pubblicazione: (2023)

Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations
di: Zheng, Mingqian, et al.
Pubblicazione: (2026)

Computational Analysis of Climate Policy
di: Hicks, Carolyn
Pubblicazione: (2025)

Nautilus Compass: Black-box Persona Drift Detection for Production LLM Agents
di: Wang, Chunxiao
Pubblicazione: (2026)

Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback
di: Zhou, Jiayi, et al.
Pubblicazione: (2024)

Hybrid Annotation for Propaganda Detection: Integrating LLM Pre-Annotations with Human Intelligence
di: Sahitaj, Ariana, et al.
Pubblicazione: (2025)

Refining and Reusing Annotation Guidelines for LLM Annotation
di: Kim, Kon Woo, et al.
Pubblicazione: (2026)

On Efficient and Statistical Quality Estimation for Data Annotation
di: Klie, Jan-Christoph, et al.
Pubblicazione: (2024)

Language Model as an Annotator: Unsupervised Context-aware Quality Phrase Generation
di: Zhang, Zhihao, et al.
Pubblicazione: (2023)

Aggregating Soft Labels from Crowd Annotations Improves Uncertainty Estimation Under Distribution Shift
di: Wright, Dustin, et al.
Pubblicazione: (2022)

Semantic Agreement Enables Efficient Open-Ended LLM Cascades
di: Soiffer, Duncan, et al.
Pubblicazione: (2025)

Quantifying the Statistical Effect of Rubric Modifications on Human-Autorater Agreement
di: Huynh, Jessica, et al.
Pubblicazione: (2026)

Disaggregation Reveals Hidden Training Dynamics: The Case of Agreement Attraction
di: Michaelov, James A., et al.
Pubblicazione: (2025)

A Novel Graph-Sequence Learning Model for Inductive Text Classification
di: Wang, Zuo, et al.
Pubblicazione: (2025)

Judge's Verdict: A Comprehensive Analysis of LLM Judge Capability Through Human Agreement
di: Han, Steve, et al.
Pubblicazione: (2025)

ReMamba: Equip Mamba with Effective Long-Sequence Modeling
di: Yuan, Danlong, et al.
Pubblicazione: (2024)

CodeBenchGen: Creating Scalable Execution-based Code Generation Benchmarks
di: Xie, Yiqing, et al.
Pubblicazione: (2024)

RepoST: Scalable Repository-Level Coding Environment Construction with Sandbox Testing
di: Xie, Yiqing, et al.
Pubblicazione: (2025)

Where is this coming from? Making groundedness count in the evaluation of Document VQA models
di: Nourbakhsh, Armineh, et al.
Pubblicazione: (2025)

CHAIR -- Classifier of Hallucination as Improver
di: Sun, Ao
Pubblicazione: (2025)