Cover Image

Human-Calibrated Automated Testing and Validation of Generative Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sudjianto, Agus, Zhang, Aijun, Neppalli, Srinivas, Joshi, Tarun, Malohlava, Michal
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Automatic Generation of Behavioral Test Cases For Natural Language Processing Using Clustering and Prompting
by: Li, Ying, et al.
Published: (2024)

Model Validation Practice in Banking: A Structured Approach for Predictive Models
by: Sudjianto, Agus, et al.
Published: (2024)

From RLHF to Direct Alignment: A Theoretical Unification of Preference Learning for Large Language Models
by: Raheja, Tarun, et al.
Published: (2026)

Human-Alignment and Calibration of Inference-Time Uncertainty in Large Language Models
by: Moore, Kyle, et al.
Published: (2025)

Large Language Models as Planning Domain Generators
by: Oswald, James, et al.
Published: (2024)

SLMEval: Entropy-Based Calibration for Human-Aligned Evaluation of Large Language Models
by: Daynauth, Roland, et al.
Published: (2025)

ATG: Benchmarking Automated Theorem Generation for Generative Language Models
by: Lin, Xiaohan, et al.
Published: (2024)

Integrating Large Language Models and Knowledge Graphs for Extraction and Validation of Textual Test Data
by: De Santis, Antonio, et al.
Published: (2024)

Risk-Averse Finetuning of Large Language Models
by: Chaudhary, Sapana, et al.
Published: (2025)

Calibration Across Layers: Understanding Calibration Evolution in LLMs
by: Joshi, Abhinav, et al.
Published: (2025)

Calibrated Language Models Must Hallucinate
by: Kalai, Adam Tauman, et al.
Published: (2023)

Think Again! The Effect of Test-Time Compute on Preferences, Opinions, and Beliefs of Large Language Models
by: Kour, George, et al.
Published: (2025)

Credence Calibration Game? Calibrating Large Language Models through Structured Play
by: Fang, Ke, et al.
Published: (2025)

SEAL: Steerable Reasoning Calibration of Large Language Models for Free
by: Chen, Runjin, et al.
Published: (2025)

Calibrating Verbalized Probabilities for Large Language Models
by: Wang, Cheng, et al.
Published: (2024)

Calibrating Reasoning in Language Models with Internal Consistency
by: Xie, Zhihui, et al.
Published: (2024)

Calibrating Large Language Models Using Their Generations Only
by: Ulmer, Dennis, et al.
Published: (2024)

Abductive Inference in Retrieval-Augmented Language Models: Generating and Validating Missing Premises
by: Lin, Shiyin
Published: (2025)

Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
by: Tian, Runchu, et al.
Published: (2025)

Generalization or Memorization? Brittleness Testing for Chess-Trained Language Models
by: Tang, Ethan
Published: (2026)

Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies
by: Ye, Yuxuan, et al.
Published: (2026)

Holistic Automated Red Teaming for Large Language Models through Top-Down Test Case Generation and Multi-turn Interaction
by: Zhang, Jinchuan, et al.
Published: (2024)

Self-training Language Models for Arithmetic Reasoning
by: Kadlčík, Marek, et al.
Published: (2024)

The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment with Large Language Models
by: Maity, Subhankar, et al.
Published: (2024)

Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models
by: Ye, Haoran, et al.
Published: (2024)

Full-ECE: A Metric For Token-level Calibration on Large Language Models
by: Liu, Han, et al.
Published: (2024)

Development and Validation of a Large Language Model for Generating Fully-Structured Radiology Reports
by: Niu, Chuang, et al.
Published: (2024)

Calibrating Long-form Generations from Large Language Models
by: Huang, Yukun, et al.
Published: (2024)

Fuzzy Fingerprinting Encoder Pre-trained Language Models for Emotion Recognition in Conversations: Human Assessment and Validity Study
by: Pereira, Patrícia, et al.
Published: (2026)

Atomic Calibration of LLMs in Long-Form Generations
by: Zhang, Caiqi, et al.
Published: (2024)

Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning
by: Zhang, Chuang, et al.
Published: (2026)

A Survey of Confidence Estimation and Calibration in Large Language Models
by: Geng, Jiahui, et al.
Published: (2023)

HypeLoRA: Hyper-Network-Generated LoRA Adapters for Calibrated Language Model Fine-Tuning
by: Trojan, Bartosz, et al.
Published: (2026)

On the Entropy Calibration of Language Models
by: Cao, Steven, et al.
Published: (2025)

Evaluating Creative Short Story Generation in Humans and Large Language Models
by: Ismayilzada, Mete, et al.
Published: (2024)

Enabling Precise Topic Alignment in Large Language Models Via Sparse Autoencoders
by: Joshi, Ananya, et al.
Published: (2025)

LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Models
by: Stengel-Eskin, Elias, et al.
Published: (2024)

Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Models
by: Huang, Liangjie, et al.
Published: (2025)

Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals
by: Torrielli, Federico, et al.
Published: (2026)

TaskBench: Benchmarking Large Language Models for Task Automation
by: Shen, Yongliang, et al.
Published: (2023)