Image de couverture de livre

Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Weng, Shihao, Feng, Yang, Xie, Xiaofei
Format:	Preprint
Publié:	2026
Sujets:	Artificial Intelligence Software Engineering
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LLM-as-a-Judge for Scalable Test Coverage Evaluation: Accuracy, Operational Reliability, and Cost
par: Huang, Donghao, et autres
Publié: (2025)

Bias Testing and Mitigation in LLM-based Code Generation
par: Huang, Dong, et autres
Publié: (2023)

The Foundation Cracks: A Comprehensive Study on Bugs and Testing Practices in LLM Libraries
par: Jiang, Weipeng, et autres
Publié: (2025)

DriveTester: A Unified Platform for Simulation-Based Autonomous Driving Testing
par: Cheng, Mingfei, et autres
Publié: (2024)

Beyond Accuracy: LLM Variability in Evidence Screening for Software Engineering SLRs
par: Hida, Gilberto Sussumu, et autres
Publié: (2026)

Beyond Accuracy: An Empirical Study on Unit Testing in Open-source Deep Learning Projects
par: Wang, Han, et autres
Publié: (2024)

Towards Reliable LLM-Driven Fuzz Testing: Vision and Road Ahead
par: Cheng, Yiran, et autres
Publié: (2025)

PentestEval: Benchmarking LLM-based Penetration Testing with Modular and Stage-Level Design
par: Yang, Ruozhao, et autres
Publié: (2025)

Evolution without an Oracle: Driving Effective Evolution with LLM Judges
par: Zhao, Zhe, et autres
Publié: (2025)

Toward Reliable Design of LLM-Enabled Agentic Workflows: Optimizing Latency-Reliability-Cost Tradeoffs
par: Yang, Ya-Ting, et autres
Publié: (2026)

Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering
par: Zhao, Zixiao, et autres
Publié: (2026)

AutoEmpirical: LLM-Based Automated Research for Empirical Software Fault Analysis
par: Yu, Jiongchi, et autres
Publié: (2025)

STCLocker: Deadlock Avoidance Testing for Autonomous Driving Systems
par: Cheng, Mingfei, et autres
Publié: (2025)

Inverting the Shield: Systematically Generating Safety Tests from Policy Specifications
par: Lu, Xiaoyue, et autres
Publié: (2026)

Beyond Accuracy: Characterizing Code Comprehension Capabilities in (Large) Language Models
par: Mächtle, Felix, et autres
Publié: (2026)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
par: Jiang, Hongchao, et autres
Publié: (2025)

Model-Enhanced LLM-Driven VUI Testing of VPA Apps
par: Li, Suwan, et autres
Publié: (2024)

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering
par: Wang, Ruiqi, et autres
Publié: (2025)

Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Code
par: Liu, Fang, et autres
Publié: (2024)

Prompt Stability in Code LLMs: Measuring Sensitivity across Emotion- and Personality-Driven Variations
par: Ma, Wei, et autres
Publié: (2025)

AXIOM: Benchmarking LLM-as-a-Judge for Code via Rule-Based Perturbation and Multisource Quality Calibration
par: Wang, Ruiqi, et autres
Publié: (2025)

LLM Company Policies and Policy Implications in Software Organizations
par: Khojah, Ranim, et autres
Publié: (2025)

Evaluating Reliability Gaps in Large Language Model Safety via Repeated Prompt Sampling
par: Broadwater, Keita
Publié: (2026)

Towards Reliable Evaluation of Neural Program Repair with Natural Robustness Testing
par: Le-Cong, Thanh, et autres
Publié: (2024)

TREAT: A Code LLMs Trustworthiness / Reliability Evaluation and Testing Framework
par: Gao, Shuzheng, et autres
Publié: (2025)

Rethinking Testing for LLM Applications: Characteristics, Challenges, and a Lightweight Interaction Protocol
par: Ma, Wei, et autres
Publié: (2025)

VerilogReader: LLM-Aided Hardware Test Generation
par: Ma, Ruiyang, et autres
Publié: (2024)

ARGUS: Defending LLM Agents Against Context-Aware Prompt Injection
par: Weng, Shihao, et autres
Publié: (2026)

A Self-Healing Framework for Reliable LLM-Based Autonomous Agents
par: Jeong, Cheonsu, et autres
Publié: (2026)

Unseen Horizons: Unveiling the Real Capability of LLM Code Generation Beyond the Familiar
par: Zhang, Yuanliang, et autres
Publié: (2024)

TENET: Leveraging Tests Beyond Validation for Code Generation
par: Hu, Yiran, et autres
Publié: (2025)

BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation
par: Lai, Peng, et autres
Publié: (2026)

MORTAR: A Model-based Runtime Action Repair Framework for AI-enabled Cyber-Physical Systems
par: Wang, Renzhi, et autres
Publié: (2024)

InfCode: Adversarial Iterative Refinement of Tests and Patches for Reliable Software Issue Resolution
par: Li, KeFan, et autres
Publié: (2025)

Engineering AI Judge Systems
par: Lin, Jiahuei, et autres
Publié: (2024)

LLM-Empowered Functional Safety and Security by Design in Automotive Systems
par: Petrovic, Nenad, et autres
Publié: (2026)

ProbGuard: Probabilistic Runtime Monitoring for LLM Agent Safety
par: Wang, Haoyu, et autres
Publié: (2025)

Tests as Prompt: A Test-Driven-Development Benchmark for LLM Code Generation
par: Cui, Yi
Publié: (2025)

Do Code Semantics Help? A Comprehensive Study on Execution Trace-Based Information for Code Large Language Models
par: Wang, Jian, et autres
Publié: (2025)

MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems
par: Jia, Jin, et autres
Publié: (2026)