Cover Image

Oracle-Checker Scheme for Evaluating a Generative Large Language Model

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zeng, Yueling Jenny, Wang, Li-C., Ibbetson, Thomas
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RefChecker: Reference-based Fine-grained Hallucination Checker and Benchmark for Large Language Models
by: Hu, Xiangkun, et al.
Published: (2024)

Are Large Language Models Table-based Fact-Checkers?
by: Zhang, Hanwen, et al.
Published: (2024)

Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models
by: Li, Miaoran, et al.
Published: (2023)

Random Testing of Model Checkers for Timed Automata with Automated Oracle Generation
by: Manini, Andrea, et al.
Published: (2025)

Interactive DualChecker for Mitigating Hallucinations in Distilling Large Language Models
by: Wang, Meiyun, et al.
Published: (2024)

ConstraintChecker: A Plugin for Large Language Models to Reason on Commonsense Knowledge Bases
by: Do, Quyet V., et al.
Published: (2024)

ConsistencyChecker: Tree-based Evaluation of LLM Generalization Capabilities
by: Hong, Zhaochen, et al.
Published: (2025)

Evaluating the Generation Capabilities of Large Chinese Language Models
by: Zeng, Hui, et al.
Published: (2023)

LLMEffiChecker: Understanding and Testing Efficiency Degradation of Large Language Models
by: Feng, Xiaoning, et al.
Published: (2022)

FoldGPT: Simple and Effective Large Language Model Compression Scheme
by: Liu, Songwei, et al.
Published: (2024)

Assessing Evaluation Metrics for Neural Test Oracle Generation
by: Shin, Jiho, et al.
Published: (2023)

Large Language Models as Biomedical Hypothesis Generators: A Comprehensive Evaluation
by: Qi, Biqing, et al.
Published: (2024)

What Makes a Medical Checker Trainable? Diagnosing Signal Collapse and Reward Hacking in Checker-Guided RAG for Biomedical QA
by: Ji, Yuelyu, et al.
Published: (2026)

Deciphering Oracle Bone Language with Diffusion Models
by: Guan, Haisu, et al.
Published: (2024)

Large Language Models on Wikipedia-Style Survey Generation: an Evaluation in NLP Concepts
by: Gao, Fan, et al.
Published: (2023)

Socrates or Smartypants: Testing Logic Reasoning Capabilities of Large Language Models with Logic Programming-based Test Oracles
by: Xu, Zihao, et al.
Published: (2025)

A Systematic Evaluation of Large Language Models for Natural Language Generation Tasks
by: Ni, Xuanfan, et al.
Published: (2024)

Evaluating Clinical Competencies of Large Language Models with a General Practice Benchmark
by: Li, Zheqing, et al.
Published: (2025)

CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation
by: Ke, Pei, et al.
Published: (2023)

Co-FactChecker: A Framework for Human-AI Collaborative Claim Verification Using Large Reasoning Models
by: Sahnan, Dhruv, et al.
Published: (2026)

A Full-duplex Speech Dialogue Scheme Based On Large Language Models
by: Wang, Peng, et al.
Published: (2024)

PsychCounsel-Bench: Evaluating the Psychology Intelligence of Large Language Models
by: Zeng, Min
Published: (2025)

Evaluating Large Language Models with Psychometrics
by: Li, Yuan, et al.
Published: (2024)

Learning Evaluation Models from Large Language Models for Sequence Generation
by: Wang, Chenglong, et al.
Published: (2023)

Vibe Checker: Aligning Code Evaluation with Human Preference
by: Zhong, Ming, et al.
Published: (2025)

Evaluating Self-Generated Documents for Enhancing Retrieval-Augmented Generation with Large Language Models
by: Li, Jiatao, et al.
Published: (2024)

OMGEval: An Open Multilingual Generative Evaluation Benchmark for Large Language Models
by: Liu, Yang, et al.
Published: (2024)

Evaluating Large Language Models at Evaluating Instruction Following
by: Zeng, Zhiyuan, et al.
Published: (2023)

The SCAN Statistical Model Checker
by: Ghiorzi, Enrico, et al.
Published: (2026)

Evaluating Accounting Reasoning Capabilities of Large Language Models
by: Zhou, Jie, et al.
Published: (2026)

Evaluating Rare Disease Diagnostic Performance in Symptom Checkers: A Synthetic Vignette Simulation Approach
by: Nishibayashi, Takashi, et al.
Published: (2025)

Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Models
by: Liu, Shuliang, et al.
Published: (2025)

Towards Data Contamination Detection for Modern Large Language Models: Limitations, Inconsistencies, and Oracle Challenges
by: Samuel, Vinay, et al.
Published: (2024)

A Fully Automated Pipeline for Conversational Discourse Annotation: Tree Scheme Generation and Labeling with Large Language Models
by: Petukhova, Kseniia, et al.
Published: (2025)

AI Analyst: Framework and Comprehensive Evaluation of Large Language Models for Financial Time Series Report Generation
by: Fons, Elizabeth, et al.
Published: (2025)

ShiZhi: A Chinese Lightweight Large Language Model for Court View Generation
by: Hou, Zhitian, et al.
Published: (2025)

Exploring Lightweight Large Language Models for Court View Generation
by: Hou, Zhitian, et al.
Published: (2026)

Rethinking Generative Large Language Model Evaluation for Semantic Comprehension
by: Wei, Fangyun, et al.
Published: (2024)

Evaluating Large Language Model Biases in Persona-Steered Generation
by: Liu, Andy, et al.
Published: (2024)

Evaluating and Improving Graph to Text Generation with Large Language Models
by: He, Jie, et al.
Published: (2025)