Cover Image

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Acharyya, Aranyak, Priebe, Carey E., Helm, Hayden S.
Format:	Preprint
Published:	2025
Subjects:	Statistics Theory Artificial Intelligence Methodology
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

When prompt perturbations break your A/B test: A valid statistical test for generative surveying
by: Helm, Hayden, et al.
Published: (2026)

Consistent estimation of generative model representations in the data kernel perspective space
by: Acharyya, Aranyak, et al.
Published: (2024)

Query-efficient model evaluation using cached responses
by: Helm, Hayden, et al.
Published: (2026)

Comparing Foundation Models using Data Kernels
by: Duderstadt, Brandon, et al.
Published: (2023)

Convergence guarantees for response prediction for latent structure network time series
by: Acharyya, Aranyak, et al.
Published: (2025)

Interaction Testing in Variation Analysis
by: Plecko, Drago
Published: (2024)

Consistent response prediction for multilayer networks on unknown manifolds
by: Acharyya, Aranyak, et al.
Published: (2024)

Hierarchical biomarker thresholding: a model-agnostic framework for stability
by: Debeaupuis, O.
Published: (2025)

Optimizing the Induced Correlation in Omnibus Joint Graph Embeddings
by: Pantazis, Konstantinos, et al.
Published: (2024)

CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency
by: Ota, Hirofumi, et al.
Published: (2026)

On the calibration of survival models with competing risks
by: Alberge, Julie, et al.
Published: (2026)

A General Framework on Conditions for Constraint-based Causal Learning
by: Teh, Kai Z., et al.
Published: (2024)

A note on incorrect inferences in non-binary qualitative probabilistic networks
by: Carter, Jack Storror
Published: (2022)

A fine-grained look at causal effects in causal spaces
by: Park, Junhyung, et al.
Published: (2025)

Correcting a Nonparametric Two-sample Graph Hypothesis Test for Graphs with Different Numbers of Vertices with Applications to Connectomics
by: Alyakin, Anton A., et al.
Published: (2020)

A novel finite-sample testing procedure for composite null hypotheses via pointwise rejection
by: Park, Joonha, et al.
Published: (2026)

A Tutorial on Asymptotic Properties for Biostatisticians with Applications to COVID-19 Data
by: Cui, Elvis Han
Published: (2022)

Federated Causal Inference from Multi-Site Observational Data via Propensity Score Aggregation
by: Khellaf, Rémi, et al.
Published: (2025)

Optimal estimation of the null distribution in large-scale inference
by: Kotekal, Subhodh, et al.
Published: (2024)

Multimodal data integration and cross-modal querying via orchestrated approximate message passing
by: Nandy, Sagnik, et al.
Published: (2024)

Bias-Aware Conformal Prediction for Metric-Based Imaging Pipelines
by: Cheung, Matt Y., et al.
Published: (2024)

LLM Web Dynamics: Tracing Model Collapse in a Network of LLMs
by: Wang, Tianyu, et al.
Published: (2025)

Recovering manifold structure in LLM responses through a joint Euclidean mirror
by: Baum, Maximilian, et al.
Published: (2026)

Revisiting Incremental Stochastic Majorization-Minimization Algorithms with Applications to Mixture of Experts
by: Tran, TrungKhang, et al.
Published: (2026)

Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processes
by: Lu, Miao, et al.
Published: (2022)

Class conditional conformal prediction for multiple inputs by p-value aggregation
by: Fermanian, Jean-Baptiste, et al.
Published: (2025)

Label Noise Robustness of Conformal Prediction
by: Einbinder, Bat-Sheva, et al.
Published: (2022)

A Likelihood Based Approach to Distribution Regression Using Conditional Deep Generative Models
by: Kumar, Shivam, et al.
Published: (2024)

Foundations of Structural Causal Models with Latent Selection
by: Chen, Leihao, et al.
Published: (2024)

Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning
by: Yu, Xiangning, et al.
Published: (2025)

Evaluating LLMs When They Do Not Know the Answer: Statistical Evaluation of Mathematical Reasoning via Comparative Signals
by: Dong, Zihan, et al.
Published: (2026)

Le Cam Distortion: A Decision-Theoretic Framework for Robust Transfer Learning
by: Akdemir, Deniz
Published: (2025)

Statistical Inference for Optimal Transport Maps: Recent Advances and Perspectives
by: Balakrishnan, Sivaraman, et al.
Published: (2025)

Conformal Risk Control
by: Angelopoulos, Anastasios N., et al.
Published: (2022)

Optimal tests of the composite null hypothesis arising in mediation analysis
by: Miles, Caleb H., et al.
Published: (2021)

Testing for similarity of dose response in multi-regional clinical trials
by: Dette, Holger, et al.
Published: (2024)

Detecting Perspective Shifts in Multi-agent Systems
by: Bridgeford, Eric, et al.
Published: (2025)

Control Charts for Multi-agent Systems
by: Helm, Hayden, et al.
Published: (2026)

False discovery proportion envelopes with m-consistency
by: Meah, Iqraa, et al.
Published: (2023)

Testing independence in the presence of missing data: high-dimensional case
by: Cuparić, Marija, et al.
Published: (2026)