Cover Image

Evaluating Large Language Models as Expert Annotators

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tseng, Yu-Min, Chen, Wei-Lin, Chen, Chung-Chi, Chen, Hsin-Hsi
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Are Expert-Level Language Models Expert-Level Annotators?
by: Tseng, Yu-Min, et al.
Published: (2024)

Hearing the Order: Investigating Position Bias in Large Audio-Language Models
by: Lin, Yu-Xiang, et al.
Published: (2025)

Diagnosing Model Editing via Knowledge Spectrum
by: Pan, Tsung-Hsuan, et al.
Published: (2025)

"Why" Has the Least Side Effect on Model Editing
by: Pan, Tsung-Hsuan, et al.
Published: (2024)

Co-Trained Retriever-Generator Framework for Question Generation in Earnings Calls
by: Juan, Yining, et al.
Published: (2024)

Modeling Professionalism in Expert Questioning through Linguistic Differentiation
by: D'Agostino, Giulia, et al.
Published: (2025)

Pre-Finetuning with Impact Duration Awareness for Stock Movement Prediction
by: Chiu, Chr-Jr, et al.
Published: (2024)

GADFA: Generator-Assisted Decision-Focused Approach for Opinion Expressing Timing Identification
by: Chen, Chung-Chi, et al.
Published: (2024)

Personalized Graph-Empowered Large Language Model for Proactive Information Access
by: Chang, Chia Cheng, et al.
Published: (2026)

Why Expert Alignment Is Hard: Evidence from Subjective Evaluation
by: Lin, Tzu-Mi, et al.
Published: (2026)

Unveiling Selection Biases: Exploring Order and Token Sensitivity in Large Language Models
by: Wei, Sheng-Lun, et al.
Published: (2024)

Rethinking Generative Large Language Model Evaluation for Semantic Comprehension
by: Wei, Fangyun, et al.
Published: (2024)

Enhancing Investment Opinion Ranking through Argument-Based Sentiment Analysis
by: Chen, Chung-Chi, et al.
Published: (2024)

Do Before You Judge: Self-Reference as a Pathway to Better LLM Evaluation
by: Lin, Wei-Hsiang, et al.
Published: (2025)

No One Fits All: From Fixed Prompting to Learned Routing in Multilingual LLMs
by: Wu, Wei-Chi, et al.
Published: (2026)

AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators
by: He, Xingwei, et al.
Published: (2023)

Judge Model for Large-scale Multimodality Benchmarks
by: Shih, Min-Han, et al.
Published: (2026)

Observing Micromotives and Macrobehavior of Large Language Models
by: Cheng, Yuyang, et al.
Published: (2024)

CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation
by: Li, Minzhi, et al.
Published: (2023)

Standardizing Longitudinal Radiology Report Evaluation via Large Language Model Annotation
by: Wang, Xinyi, et al.
Published: (2026)

ANAH: Analytical Annotation of Hallucinations in Large Language Models
by: Ji, Ziwei, et al.
Published: (2024)

Enhancing Financial Sentiment Analysis with Expert-Designed Hint
by: Chen, Chung-Chi, et al.
Published: (2024)

Refining Financial Consumer Complaints through Multi-Scale Model Interaction
by: Chen, Bo-Wei, et al.
Published: (2025)

Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inference
by: Chen, Bo-Wei, et al.
Published: (2026)

SciDFM: A Large Language Model with Mixture-of-Experts for Science
by: Sun, Liangtai, et al.
Published: (2024)

Evaluating and Steering Modality Preferences in Multimodal Large Language Model
by: Zhang, Yu, et al.
Published: (2025)

Selective Annotation via Data Allocation: These Data Should Be Triaged to Experts for Annotation Rather Than the Model
by: Huang, Chen, et al.
Published: (2024)

Decision-Oriented Text Evaluation
by: Huang, Yu-Shiang, et al.
Published: (2025)

Self-guided Knowledgeable Network of Thoughts: Amplifying Reasoning with Large Language Models
by: Chen, Chao-Chi, et al.
Published: (2024)

Automating Expert-Level Medical Reasoning Evaluation of Large Language Models
by: Zhou, Shuang, et al.
Published: (2025)

IDEAlign: Comparing Large Language Models to Human Experts in Open-ended Interpretive Annotations
by: Nam, Hyunji, et al.
Published: (2025)

XIFBench: Evaluating Large Language Models on Multilingual Instruction Following
by: Li, Zhenyu, et al.
Published: (2025)

AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning
by: Lin, Tzu-Han, et al.
Published: (2025)

Automated Privacy Information Annotation in Large Language Model Interactions
by: Zeng, Hang, et al.
Published: (2025)

Fake News Detection: Comparative Evaluation of BERT-like Models and Large Language Models with Generative AI-Annotated Data
by: Raza, Shaina, et al.
Published: (2024)

From Facts to Insights: A Study on the Generation and Evaluation of Analytical Reports for Deciphering Earnings Calls
by: Goldsack, Tomas, et al.
Published: (2024)

Aligning Large Language Models via Fully Self-Synthetic Data
by: Yin, Shangjian, et al.
Published: (2025)

ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models
by: Gu, Yuzhe, et al.
Published: (2024)

Beyond Turing Test: Can GPT-4 Sway Experts' Decisions?
by: Takayanagi, Takehiro, et al.
Published: (2024)

Evaluating Cultural Knowledge Processing in Large Language Models: A Cognitive Benchmarking Framework Integrating Retrieval-Augmented Generation
by: Lee, Hung-Shin, et al.
Published: (2025)