Buchumschlag

Large Language Models are Inconsistent and Biased Evaluators

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Stureborg, Rickard, Alikaniotis, Dimitris, Suhara, Yoshi
Format:	Preprint
Veröffentlicht:	2024
Schlagworte:	Computation and Language Artificial Intelligence 68T50 (Primary) 68T01, 68T37, 91F20 (Secondary) I.2; I.2.7; I.7
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Ähnliche Einträge

Tailoring Vaccine Messaging with Common-Ground Opinions
von: Stureborg, Rickard, et al.
Veröffentlicht: (2024)

BEATS: Bias Evaluation and Assessment Test Suite for Large Language Models
von: Abhishek, Alok, et al.
Veröffentlicht: (2025)

SHARP: Social Harm Analysis via Risk Profiles for Measuring Inequities in Large Language Models
von: Abhishek, Alok, et al.
Veröffentlicht: (2026)

Data and AI governance: Promoting equity, ethics, and fairness in large language models
von: Abhishek, Alok, et al.
Veröffentlicht: (2025)

Reasoning Promotes Robustness in Theory of Mind Tasks
von: de Haan, Ian B., et al.
Veröffentlicht: (2026)

Smotrom tvoja pa ander drogoj verden! Resurrecting Dead Pidgin with Generative Models: Russenorsk Case Study
von: Tikhonov, Alexey, et al.
Veröffentlicht: (2025)

Extracting Abstraction Dimensions by Identifying Syntax Pattern from Texts
von: Zhou, Jian, et al.
Veröffentlicht: (2025)

Unveiling factors influencing judgment variation in Sentiment Analysis with Natural Language Processing and Statistics
von: Kellert, Olga, et al.
Veröffentlicht: (2024)

Approaches to Semantic Textual Similarity in Slovak Language: From Algorithms to Transformers
von: Radosky, Lukas, et al.
Veröffentlicht: (2026)

MORQA: Benchmarking Evaluation Metrics for Medical Open-Ended Question Answering
von: Yim, Wen-wai, et al.
Veröffentlicht: (2025)

Towards Probabilistic Question Answering Over Tabular Data
von: Shen, Chen, et al.
Veröffentlicht: (2025)

Linguistic Collapse: Neural Collapse in (Large) Language Models
von: Wu, Robert, et al.
Veröffentlicht: (2024)

CogniLoad: A Synthetic Natural Language Reasoning Benchmark With Tunable Length, Intrinsic Difficulty, and Distractor Density
von: Kaiser, Daniel, et al.
Veröffentlicht: (2025)

CLEV: LLM-Based Evaluation Through Lightweight Efficient Voting for Free-Form Question-Answering
von: Badshah, Sher, et al.
Veröffentlicht: (2025)

An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT
von: Ma, Chong, et al.
Veröffentlicht: (2023)

TRAWL: Tensor Reduced and Approximated Weights for Large Language Models
von: Luo, Yiran, et al.
Veröffentlicht: (2024)

Increasing the Difficulty of Automatically Generated Questions via Reinforcement Learning with Synthetic Preference
von: Thorne, William, et al.
Veröffentlicht: (2024)

Project Synapse: A Hierarchical Multi-Agent Framework with Hybrid Memory for Autonomous Resolution of Last-Mile Delivery Disruptions
von: Yadav, Arin Gopalan, et al.
Veröffentlicht: (2026)

A Collaborative Content Moderation Framework for Toxicity Detection based on Conformalized Estimates of Annotation Disagreement
von: Villate-Castillo, Guillermo, et al.
Veröffentlicht: (2024)

An Epidemiological Knowledge Graph extracted from the World Health Organization's Disease Outbreak News
von: Consoli, Sergio, et al.
Veröffentlicht: (2025)

Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form QA
von: Badshah, Sher, et al.
Veröffentlicht: (2024)

A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthiness
von: Wang, Fali, et al.
Veröffentlicht: (2025)

TableMoE: Neuro-Symbolic Routing for Structured Expert Reasoning in Multimodal Table Understanding
von: Zhang, Junwen, et al.
Veröffentlicht: (2025)

Co-NAML-LSTUR: A Combined Model with Attentive Multi-View Learning and Long- and Short-term User Representations for News Recommendation
von: Nguyen, Minh Hoang, et al.
Veröffentlicht: (2025)

On measuring grounding and generalizing grounding problems
von: Quigley, Daniel, et al.
Veröffentlicht: (2025)

Vibe-Creation: The Epistemology of Human-AI Emergent Cognition
von: Levin, Ilya
Veröffentlicht: (2026)

InhibiDistilbert: Knowledge Distillation for a ReLU and Addition-based Transformer
von: Zhang, Tony, et al.
Veröffentlicht: (2025)

Low-Resource Neural Machine Translation Using Recurrent Neural Networks and Transfer Learning: A Case Study on English-to-Igbo
von: Ekle, Ocheme Anthony, et al.
Veröffentlicht: (2025)

Large Language Models Report Subjective Experience Under Self-Referential Processing
von: Berg, Cameron, et al.
Veröffentlicht: (2025)

Do Reasoning Models Enhance Embedding Models?
von: Chan, Wun Yu, et al.
Veröffentlicht: (2026)

Research on a hybrid LSTM-CNN-Attention model for text-based web content classification
von: Kuz, Mykola, et al.
Veröffentlicht: (2025)

GLEAN: Grounded Lightweight Evaluation Anchors for Contamination-Aware Tabular Reasoning
von: Wang, Qizhi
Veröffentlicht: (2026)

Epidemic Information Extraction for Event-Based Surveillance using Large Language Models
von: Consoli, Sergio, et al.
Veröffentlicht: (2024)

Uncovering Uncertainty in Transformer Inference
von: Brothers, Greyson, et al.
Veröffentlicht: (2024)

What is lost in Normalization? Exploring Pitfalls in Multilingual ASR Model Evaluations
von: Manohar, Kavya, et al.
Veröffentlicht: (2024)

A transfer learning approach for automatic conflicts detection in software requirement sentence pairs based on dual encoders
von: Wang, Yizheng, et al.
Veröffentlicht: (2025)

Rule Extraction in Machine Learning: Chat Incremental Pattern Constructor
von: Nwokocha, Caleb Princewill
Veröffentlicht: (2022)

JAM: Controllable and Responsible Text Generation via Causal Reasoning and Latent Vector Manipulation
von: Huang, Yingbing, et al.
Veröffentlicht: (2025)

Solving Zebra Puzzles Using Constraint-Guided Multi-Agent Systems
von: Berman, Shmuel, et al.
Veröffentlicht: (2024)

PaperAudit-Bench: Benchmarking Error Detection in Research Papers for Critical Automated Peer Review
von: Tu, Songjun, et al.
Veröffentlicht: (2026)