Cover Image

Distribution-Free Uncertainty Quantification for Continuous AI Agent Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gao, Yuxuan, Wang, Megan, Yu, Yi Ling
Format:	Preprint
Published:	2026
Subjects:	Artificial Intelligence Machine Learning 62G15, 62L12 I.2.6; I.2.7; G.3
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ICE: Intervention-Consistent Explanation Evaluation with Statistical Grounding for LLMs
by: Basu, Abhinaba, et al.
Published: (2026)

Conformal Path Reasoning: Trustworthy Knowledge Graph Question Answering via Path-Level Calibration
by: Lin, Shuhang, et al.
Published: (2026)

Latent-Autoregressive GP-VAE Language Model
by: Ruffenach, Yves
Published: (2025)

Bayesian Generalized Nonlinear Models Offer Basis Free SINDy With Model Uncertainty
by: Hubin, Aliaksandr
Published: (2025)

SCOPE: Selective Conformal Optimized Pairwise LLM Judging
by: Badshah, Sher, et al.
Published: (2026)

Conformal Prediction Sets for Next-Token Prediction in Large Language Models: Balancing Coverage Guarantees with Set Efficiency
by: Kotla, Yoshith Roy, et al.
Published: (2025)

Explainable Bayesian deep learning through input-skip Latent Binary Bayesian Neural Networks
by: Høyheim, Eirik, et al.
Published: (2025)

FBMS: An R Package for Flexible Bayesian Model Selection and Model Averaging
by: Frommlet, Florian, et al.
Published: (2025)

From Tail Universality to Bernstein-von Mises: A Unified Statistical Theory of Semi-Implicit Variational Inference
by: Plummer, Sean
Published: (2025)

Bias-Corrected Adaptive Conformal Inference for Multi-Horizon Time Series Forecasting
by: Lade, Ankit, et al.
Published: (2026)

Higher-Order Equilibrium Tracking for EM-Compressible Online Estimation
by: Li, ZhiMing, et al.
Published: (2026)

Constant-Target Energy Matching: A Unified Framework for Continuous and Discrete Density Estimation
by: Zeng, Zhijun, et al.
Published: (2026)

Evaluating the Quality of the Quantified Uncertainty for (Re)Calibration of Data-Driven Regression Models
by: Wibbeke, Jelke, et al.
Published: (2025)

MEDLEY-BENCH: Scale Buys Evaluation but Not Control in AI Metacognition
by: Abtahi, Farhad, et al.
Published: (2026)

Amortized Variational Inference for Logistic Regression with Missing Covariates
by: Cherifi, M., et al.
Published: (2026)

When to Trust Confidence Thresholding: Calibration Diagnostics for Pseudo-Labelled Regression
by: Kurbucz, Marcell T.
Published: (2026)

ValueBlindBench: Agreement-Gated Stress Testing of LLM-Judged Investment Rationales Before Returns Are Observable
by: Chang, Sidi, et al.
Published: (2026)

Deep Jump Gaussian Processes for Surrogate Modeling of High-Dimensional Piecewise Continuous Functions
by: Xu, Yang, et al.
Published: (2025)

Support-Conditioned Flow Matching Is Kernel Smoothing
by: Smola, Daniel Matsui
Published: (2026)

A Hybrid Framework for Real-Time Data Drift and Anomaly Identification Using Hierarchical Temporal Memory and Statistical Tests
by: Bandyopadhyay, Subhadip, et al.
Published: (2025)

From Noisy News Sentiment Scores to Interpretable Temporal Dynamics: A Bayesian State-Space Model
by: Casals, Ian Carbó
Published: (2026)

AgentPulse: A Continuous Multi-Signal Framework for Evaluating AI Agents in Deployment
by: Gao, Yuxuan, et al.
Published: (2026)

Calibrated Counterfactual Conformal Fairness ($C^3F$): Post-hoc, Shift-Aware Coverage Parity via Conformal Prediction and Counterfactual Regularization
by: Alpay, Faruk, et al.
Published: (2025)

When Are Two RLHF Objectives the Same?
by: Gaikwad, Madhava
Published: (2025)

SHIFT: Robust Double Machine Learning for Average Dose-Response Functions under Heavy-Tailed Contamination
by: Uehara, Eichi
Published: (2026)

Sequential Monte Carlo Bandits
by: Urteaga, Iñigo, et al.
Published: (2018)

Machine Collaboration
by: Liu, Qingfeng, et al.
Published: (2021)

Signature-Kernel Based Evaluation Metrics for Robust Probabilistic and Tail-Event Forecasting
by: Redhead, Benjamin R., et al.
Published: (2026)

Online Statistical Inference of Constant Sample-averaged Q-Learning
by: Panda, Saunak Kumar, et al.
Published: (2026)

Forecasting intermittent time series with Gaussian Processes and Tweedie likelihood
by: Damato, Stefano, et al.
Published: (2025)

CDFlow: Building Invertible Layers with Circulant and Diagonal Matrices
by: Feng, Xuchen, et al.
Published: (2025)

Conservative Decisions with Risk Scores
by: Wei, Yishu, et al.
Published: (2025)

A General Framework for Off-Policy Learning with Partially-Observed Reward
by: Takehi, Rikiya, et al.
Published: (2025)

metabeta -- A fast neural model for Bayesian mixed-effects regression
by: Kipnis, Alex, et al.
Published: (2025)

Theoretical Foundations of Latent Posterior Factors: Formal Guarantees for Multi-Evidence Reasoning
by: Alege, Aliyu Agboola
Published: (2026)

Statistical Measures for Explainable Aspect-Based Sentiment Analysis: A Case Study on Environmental Discourse in Reddit
by: Stracqualursi, Luisa, et al.
Published: (2026)

Empirical Decision Theory
by: Jansen, Christoph, et al.
Published: (2025)

Improving ML Training Data with Gold-Standard Quality Metrics
by: Barrett, Leslie, et al.
Published: (2025)

Is Causality Necessary for Efficient Portfolios? A Computational Perspective on Predictive Validity and Model Misspecification
by: Dominguez, Alejandro Rodriguez
Published: (2025)

The Unified Non-Convex Framework for Robust Causal Inference: Overcoming the Gaussian Barrier and Optimization Fragility
by: Uehara, Eichi
Published: (2025)