Cover Image

CONTESTS: a Framework for Consistency Testing of Span Probabilities in Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wagner, Eitan, Slavutsky, Yuli, Abend, Omri
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Express Your Doubts -- Probabilistic World Modeling Should not be Based on Token logprobs
by: Wagner, Eitan, et al.
Published: (2025)

Exploring the Learning Capabilities of Language Models using LEVERWORLDS
by: Wagner, Eitan, et al.
Published: (2024)

Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning
by: Wagner, Eitan, et al.
Published: (2024)

Measuring Pragmatic Influence in Large Language Model Instructions
by: Geng, Yilin, et al.
Published: (2026)

Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMs
by: Ifergan, Maxim, et al.
Published: (2024)

The Challenge and Reward of Fair Play in Narrative: A Computational Approach
by: Wagner, Eitan, et al.
Published: (2025)

A Language-agnostic Model of Child Language Acquisition
by: Mahon, Louis, et al.
Published: (2024)

Identifying Narrative Patterns and Outliers in Holocaust Testimonies Using Topic Modeling
by: Ifergan, Maxim, et al.
Published: (2024)

Unsupervised Location Mapping for Narrative Corpora
by: Wagner, Eitan, et al.
Published: (2025)

Unsupervised Speech Segmentation: A General Approach Using Speech Language Models
by: Elmakies, Avishai, et al.
Published: (2025)

A Nurse is Blue and Elephant is Rugby: Cross Domain Alignment in Large Language Models Reveal Human-like Patterns
by: Yehudai, Asaf, et al.
Published: (2024)

Computational Analysis of Character Development in Holocaust Testimonies
by: Shizgal, Esther, et al.
Published: (2024)

Control Illusion: The Failure of Instruction Hierarchies in Large Language Models
by: Geng, Yilin, et al.
Published: (2025)

Locally Measuring Cross-lingual Lexical Alignment: A Domain and Word Level Perspective
by: Karidi, Taelin, et al.
Published: (2024)

Generating Benchmarks for Factuality Evaluation of Language Models
by: Muhlgay, Dor, et al.
Published: (2023)

Predicting Classification Accuracy When Adding New Unobserved Classes
by: Slavutsky, Yuli, et al.
Published: (2020)

Toward a Benchmark for Controllable Simulation of Imperfect Students with Large Language Models
by: Apartsin, Alexander, et al.
Published: (2026)

Self-Selected Attention Span for Accelerating Large Language Model Inference
by: Jin, Tian, et al.
Published: (2024)

Exploring the Performance of Large Language Models on Subjective Span Identification Tasks
by: Dmonte, Alphaeus, et al.
Published: (2026)

Knowledge-based Consistency Testing of Large Language Models
by: Rajan, Sai Sathiesh, et al.
Published: (2024)

Calibrating Verbalized Probabilities for Large Language Models
by: Wang, Cheng, et al.
Published: (2024)

Incoherent Probability Judgments in Large Language Models
by: Zhu, Jian-Qiao, et al.
Published: (2024)

StructLens: A Structural Lens for Language Models via Maximum Spanning Trees
by: Sakajo, Haruki, et al.
Published: (2026)

DLP: Dynamic Layerwise Pruning in Large Language Models
by: Chen, Yuli, et al.
Published: (2025)

CLLMs: Consistency Large Language Models
by: Kou, Siqi, et al.
Published: (2024)

Language Models can Evaluate Themselves via Probability Discrepancy
by: Xia, Tingyu, et al.
Published: (2024)

ConSCompF: Consistency-focused Similarity Comparison Framework for Generative Large Language Models
by: Karev, Alexey, et al.
Published: (2025)

Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models
by: Yu, Zhiwei, et al.
Published: (2025)

Position Paper On Diagnostic Uncertainty Estimation from Large Language Models: Next-Word Probability Is Not Pre-test Probability
by: Gao, Yanjun, et al.
Published: (2024)

$T^5Score$: A Methodology for Automatically Assessing the Quality of LLM Generated Multi-Document Topic Sets
by: Trainin, Itamar, et al.
Published: (2024)

Calibrating Reasoning in Language Models with Internal Consistency
by: Xie, Zhihui, et al.
Published: (2024)

Post-Training Language Models for Crosslingual Consistency
by: Liu, Tianyu, et al.
Published: (2026)

DCR-Consistency: Divide-Conquer-Reasoning for Consistency Evaluation and Improvement of Large Language Models
by: Cui, Wendi, et al.
Published: (2024)

A Content-Based Framework for Cybersecurity Refusal Decisions in Large Language Models
by: Linder, Noa, et al.
Published: (2026)

Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Models
by: Kauf, Carina, et al.
Published: (2024)

Cross-Modal Consistency in Multimodal Large Language Models
by: Zhang, Xiang, et al.
Published: (2024)

Evaluating Consistency and Reasoning Capabilities of Large Language Models
by: Saxena, Yash, et al.
Published: (2024)

Recovering Event Probabilities from Large Language Model Embeddings via Axiomatic Constraints
by: Zhu, Jian-Qiao, et al.
Published: (2025)

Scaling Bidirectional Spans and Span Violations in Attention Mechanism
by: Kim, Jongwook, et al.
Published: (2025)

Harnessing Consistency for Robust Test-Time LLM Ensemble
by: Zeng, Zhichen, et al.
Published: (2025)