Cover Image

Why Expert Alignment Is Hard: Evidence from Subjective Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lin, Tzu-Mi, Hirota, Wataru, Ishigaki, Tatsuya, Lee, Lung-Hao, Chen, Chung-Chi
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Aggregate vs. Personalized Judges in Business Idea Evaluation: Evidence from Expert Disagreement
by: Hirota, Wataru, et al.
Published: (2026)

Exploring Design of Multi-Agent LLM Dialogues for Research Ideation
by: Ueda, Keisuke, et al.
Published: (2025)

Evaluating Large Language Models as Expert Annotators
by: Tseng, Yu-Min, et al.
Published: (2025)

Multimodal Task Interference: A Benchmark and Analysis of History-Target Mismatch in Multimodal LLMs
by: Kawarada, Masayuki, et al.
Published: (2026)

Are Expert-Level Language Models Expert-Level Annotators?
by: Tseng, Yu-Min, et al.
Published: (2024)

Prompting for Numerical Sequences: A Case Study on Market Comment Generation
by: Kawarada, Masayuki, et al.
Published: (2024)

Modeling Professionalism in Expert Questioning through Linguistic Differentiation
by: D'Agostino, Giulia, et al.
Published: (2025)

Enhancing Financial Sentiment Analysis with Expert-Designed Hint
by: Chen, Chung-Chi, et al.
Published: (2024)

"Why" Has the Least Side Effect on Model Editing
by: Pan, Tsung-Hsuan, et al.
Published: (2024)

A Comparative Study of Demonstration Selection for Practical Large Language Models-based Next POI Prediction
by: Nishida, Ryo, et al.
Published: (2026)

Recurrent Alignment with Hard Attention for Hierarchical Text Rating
by: Lin, Chenxi, et al.
Published: (2024)

From Facts to Insights: A Study on the Generation and Evaluation of Analytical Reports for Deciphering Earnings Calls
by: Goldsack, Tomas, et al.
Published: (2024)

Pretraining and Updates of Domain-Specific LLM: A Case Study in the Japanese Business Domain
by: Takahashi, Kosuke, et al.
Published: (2024)

Beyond Turing Test: Can GPT-4 Sway Experts' Decisions?
by: Takayanagi, Takehiro, et al.
Published: (2024)

Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment
by: Liu, Zhili, et al.
Published: (2024)

Decision-Oriented Text Evaluation
by: Huang, Yu-Shiang, et al.
Published: (2025)

The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs
by: Baeumel, Tanja, et al.
Published: (2025)

An Effective Mixture-Of-Experts Approach For Code-Switching Speech Recognition Leveraging Encoder Disentanglement
by: Yang, Tzu-Ting, et al.
Published: (2024)

Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas
by: Chen, Shiqi, et al.
Published: (2025)

Enhancing Medication Recommendation with LLM Text Representation
by: Lee, Yu-Tzu
Published: (2024)

When Silence Matters: The Impact of Irrelevant Audio on Text Reasoning in Large Audio-Language Models
by: Li, Chen-An, et al.
Published: (2025)

VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech
by: Lin, Yi-Cheng, et al.
Published: (2026)

Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essays
by: Lee, Jinsook, et al.
Published: (2025)

DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging
by: Lin, Tzu-Han, et al.
Published: (2024)

Not All Subjectivity Is the Same! Defining Desiderata for the Evaluation of Subjectivity in NLP
by: Khurana, Urja, et al.
Published: (2026)

DEER: A Benchmark for Evaluating Deep Research Agents on Expert Report Generation
by: Han, Janghoon, et al.
Published: (2025)

Schema Lineage Extraction at Scale: Multilingual Pipelines, Composite Evaluation, and Language-Model Benchmarks
by: Yin, Jiaqi, et al.
Published: (2025)

FinNuE: Exposing the Risks of Using BERTScore for Numerical Semantic Evaluation in Finance
by: Huang, Yu-Shiang, et al.
Published: (2025)

Routing Absorption in Sparse Attention: Why Random Gates Are Hard to Beat
by: Aquino-Michaels, Keston
Published: (2026)

Towards Efficient Visual-Language Alignment of the Q-Former for Visual Reasoning Tasks
by: Kim, Sungkyung, et al.
Published: (2024)

PIKA: Expert-Level Synthetic Datasets for Post-Training Alignment from Scratch
by: Yin, Shangjian, et al.
Published: (2025)

Detection Is Cheap, Routing Is Learned: Why Refusal-Based Alignment Evaluation Fails
by: Frank, Gregory N.
Published: (2026)

MelHuBERT: A simplified HuBERT on Mel spectrograms
by: Lin, Tzu-Quan, et al.
Published: (2022)

Real-Time Generation of Game Video Commentary with Multimodal LLMs: Pause-Aware Decoding Approaches
by: Afzal, Anum, et al.
Published: (2026)

Commitment Checklist: Auditing Author Commitments in Peer Review
by: Chen, Chung-Chi, et al.
Published: (2026)

AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning
by: Lin, Tzu-Han, et al.
Published: (2025)

MedLayBench-V: A Large-Scale Benchmark for Expert-Lay Semantic Alignment in Medical Vision Language Models
by: Jang, Han, et al.
Published: (2026)

Parameter Alignment Mitigates Catastrophic Forgetting in Multilingual Expert Language Models
by: Ahuja, Sanchit, et al.
Published: (2026)

Why Is RLHF Alignment Shallow? A Gradient Analysis
by: Young, Robin
Published: (2026)

Why Do We Laugh? Annotation and Taxonomy Generation for Laughable Contexts in Spontaneous Text Conversation
by: Inoue, Koji, et al.
Published: (2025)