Imagen de Portada

Consistency as a Testable Property: Statistical Methods to Evaluate AI Agent Reliability

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Raj, Harsh, Orkat, Niranjan, Mukherjee, Suvrorup, Guha, Aritra, Flynn, Cheryl, Majumdar, Subhabrata
Formato:	Preprint
Publicado:	2026
Materias:	Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Semantic Consistency for Assuring Reliability of Large Language Models
por: Raj, Harsh, et al.
Publicado: (2023)

Red Teaming AI Red Teaming
por: Majumdar, Subhabrata, et al.
Publicado: (2025)

Consistency in Language Models: Current Landscape, Challenges, and Future Directions
por: Novikova, Jekaterina, et al.
Publicado: (2025)

Improving Consistency in Large Language Models through Chain of Guidance
por: Raj, Harsh, et al.
Publicado: (2025)

Can LLMs replace Neil deGrasse Tyson? Evaluating the Reliability of LLMs as Science Communicators
por: Bajpai, Prasoon, et al.
Publicado: (2024)

Emergence WebVoyager: Toward Consistent and Transparent Evaluation of (Web) Agents in The Wild
por: Akkil, Deepak, et al.
Publicado: (2026)

AgentEval: Generative Agents as Reliable Proxies for Human Evaluation of AI-Generated Content
por: Vu, Thanh, et al.
Publicado: (2025)

Agentic AI in Healthcare & Medicine: A Seven-Dimensional Taxonomy for Empirical Evaluation of LLM-based Agents
por: Vatsal, Shubham, et al.
Publicado: (2026)

Localized LoRA: A Structured Low-Rank Approximation for Efficient Fine-Tuning
por: Barazandeh, Babak, et al.
Publicado: (2025)

Off-Policy Evaluation and Counterfactual Methods in Dynamic Auction Environments
por: Guha, Ritam, et al.
Publicado: (2025)

Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation
por: Harsh, Reetu Raj, et al.
Publicado: (2026)

MIXRAG : Mixture-of-Experts Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering
por: Liu, Lihui, et al.
Publicado: (2025)

Black-Box Reliability Certification for AI Agents via Self-Consistency Sampling and Conformal Calibration
por: Mouzouni, Charafeddine
Publicado: (2026)

Demystifying ChatGPT: How It Masters Genre Recognition
por: Raj, Subham, et al.
Publicado: (2025)

Generative Agent-Based Modeling: Unveiling Social System Dynamics through Coupling Mechanistic Models with Generative Artificial Intelligence
por: Ghaffarzadegan, Navid, et al.
Publicado: (2023)

Statistical Methods in Generative AI
por: Dobriban, Edgar
Publicado: (2025)

ORION: Teaching Language Models to Reason Efficiently in the Language of Thought
por: Tanmay, Kumar, et al.
Publicado: (2025)

Towards a Science of AI Agent Reliability
por: Rabanser, Stephan, et al.
Publicado: (2026)

Reliable and Scalable Robot Policy Evaluation with Imperfect Simulators
por: Badithela, Apurva, et al.
Publicado: (2025)

Literary Narrative as Moral Probe : A Cross-System Framework for Evaluating AI Ethical Reasoning and Refusal Behavior
por: Flynn, David C.
Publicado: (2026)

On the Reliability of AI Methods in Drug Discovery: Evaluation of Boltz-2 for Structure and Binding Affinity Prediction
por: Wan, Shunzhou, et al.
Publicado: (2026)

Certificates without Electrons? Theory and Evidence on Impacts from AI-Driven Power Demand
por: Golden, Dana, et al.
Publicado: (2026)

Improving Physics Reasoning in Large Language Models Using Mixture of Refinement Agents
por: Jaiswal, Raj, et al.
Publicado: (2024)

ParaCodex: A Profiling-Guided Autonomous Coding Agent for Reliable Parallel Code Generation and Translation
por: Kaplan, Erel, et al.
Publicado: (2026)

NLKI: A lightweight Natural Language Knowledge Integration Framework for Improving Small VLMs in Commonsense VQA Tasks
por: Dutta, Aritra, et al.
Publicado: (2025)

STED and Consistency Scoring: A Framework for Evaluating LLM Structured Output Reliability
por: Wang, Guanghui, et al.
Publicado: (2025)

Multi-Agent Conformal Prediction with Personalized Statistical Validity
por: Vejling, Martin V., et al.
Publicado: (2026)

Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents
por: Li, Miles Q., et al.
Publicado: (2026)

Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation
por: Kapoor, Sayash, et al.
Publicado: (2025)

ManifoldKV: Training-Free KV Cache Compression via Euclidean Outlier Detection
por: Datta, Debajyoti, et al.
Publicado: (2026)

A Framework for Evaluating Emerging Cyberattack Capabilities of AI
por: Rodriguez, Mikel, et al.
Publicado: (2025)

Improving Score Reliability of Multiple Choice Benchmarks with Consistency Evaluation and Altered Answer Choices
por: Cavalin, Paulo, et al.
Publicado: (2025)

From Ethical Declarations to Provable Independence: An Ontology-Driven Optimal-Transport Framework for Certifiably Fair AI Systems
por: Bhattacharya, Sukriti, et al.
Publicado: (2025)

A Testable Certificate for Constant Collapse in Teacher-Guided VAEs
por: Zhang, Zegu, et al.
Publicado: (2026)

Explanation Beyond Intuition: A Testable Criterion for Inherent Explainability
por: Merry, Michael, et al.
Publicado: (2025)

On the Reliability of Computer Use Agents
por: Gonzalez-Pumariega, Gonzalo, et al.
Publicado: (2026)

BioAgent Bench: An AI Agent Evaluation Suite for Bioinformatics
por: Fa, Dionizije, et al.
Publicado: (2026)

AgentCompass: Towards Reliable Evaluation of Agentic Workflows in Production
por: Kartik, NVJK, et al.
Publicado: (2025)

Analyzing Probabilistic Methods for Evaluating Agent Capabilities
por: Højmark, Axel, et al.
Publicado: (2024)

How Reliable Are Automatic Evaluation Methods for Instruction-Tuned LLMs?
por: Doostmohammadi, Ehsan, et al.
Publicado: (2024)