Buchumschlag

Evaluating Reliability Gaps in Large Language Model Safety via Repeated Prompt Sampling

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
1. Verfasser:	Broadwater, Keita
Format:	Preprint
Veröffentlicht:	2026
Schlagworte:	Artificial Intelligence Software Engineering
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Ähnliche Einträge

Evaluating LLM Safety Under Repeated Inference via Accelerated Prompt Stress Testing
von: Broadwater, Keita
Veröffentlicht: (2026)

Accuracy, Stability, and Repeated-Run Reliability of Large Language Models on Deterministic Programming Tasks
von: Zhou, Yongxi, et al.
Veröffentlicht: (2026)

A Hierarchical Imprecise Probability Approach to Reliability Assessment of Large Language Models
von: Aghazadeh-Chakherlou, Robab, et al.
Veröffentlicht: (2025)

Conventional Commit Classification using Large Language Models and Prompt Engineering
von: Quadir, H. M. Sazzad, et al.
Veröffentlicht: (2026)

INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair
von: Wang, Hanbin, et al.
Veröffentlicht: (2023)

Evaluating Large Language Models for Code Review
von: Cihan, Umut, et al.
Veröffentlicht: (2025)

PromptPex: Automatic Test Generation for Language Model Prompts
von: Sharma, Reshabh K, et al.
Veröffentlicht: (2025)

Welcome Your New AI Teammate: On Safety Analysis by Leashing Large Language Models
von: Nouri, Ali, et al.
Veröffentlicht: (2024)

Generating Automotive Code: Large Language Models for Software Development and Verification in Safety-Critical Systems
von: Kirchner, Sven, et al.
Veröffentlicht: (2025)

Bridging the Gap: Empowering Small Models in Reliable OpenACC-based Parallelization via GEPA-Optimized Prompting
von: Jhaveri, Samyak, et al.
Veröffentlicht: (2026)

Evaluating Large Language Models for Detecting Architectural Decision Violations
von: Su, Ruoyu, et al.
Veröffentlicht: (2026)

CAKE: Cloud Architecture Knowledge Evaluation of Large Language Models
von: Adam, Tim Lukas, et al.
Veröffentlicht: (2026)

CrossPL: Evaluating Large Language Models on Cross Programming Language Code Generation
von: Xiong, Zhanhang, et al.
Veröffentlicht: (2025)

On the Challenges of Fuzzing Techniques via Large Language Models
von: Huang, Linghan, et al.
Veröffentlicht: (2024)

Multi-Sample Prompting and Actor-Critic Prompt Optimization for Diverse Synthetic Data Generation
von: El-Hajjami, Abdelkarim, et al.
Veröffentlicht: (2025)

Large Language Models as Test Case Generators: Performance Evaluation and Enhancement
von: Li, Kefan, et al.
Veröffentlicht: (2024)

Output Format Biases in the Evaluation of Large Language Models for Code Translation
von: Macedo, Marcos, et al.
Veröffentlicht: (2024)

Legal Compliance Evaluation of Smart Contracts Generated By Large Language Models
von: Wijayakoon, Chanuka, et al.
Veröffentlicht: (2025)

Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges
von: Weng, Shihao, et al.
Veröffentlicht: (2026)

Prompt Migration: Stabilizing GenAI Applications with Evolving Large Language Models
von: Tripathi, Shivani, et al.
Veröffentlicht: (2025)

Impact of Code Context and Prompting Strategies on Automated Unit Test Generation with Modern General-Purpose Large Language Models
von: Walczak, Jakub, et al.
Veröffentlicht: (2025)

RepoDebug: Repository-Level Multi-Task and Multi-Language Debugging Evaluation of Large Language Models
von: Liu, Jingjing, et al.
Veröffentlicht: (2025)

CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation
von: Du, Kounianhua, et al.
Veröffentlicht: (2024)

AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation
von: Zhang, Tanghaoran, et al.
Veröffentlicht: (2026)

COMPASS: A Multi-Dimensional Benchmark for Evaluating Code Generation in Large Language Models
von: Meaden, James, et al.
Veröffentlicht: (2025)

SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models
von: Xu, Jingxuan, et al.
Veröffentlicht: (2025)

Evaluating Large Language Models for the Generation of Unit Tests with Equivalence Partitions and Boundary Values
von: Rodríguez, Martín, et al.
Veröffentlicht: (2025)

Evaluating Large Language Models in Code Generation: INFINITE Methodology for Defining the Inference Index
von: Christakis, Nicholas, et al.
Veröffentlicht: (2025)

CodeGolf Bench: A Multi-Language Benchmark for Evaluating Concise Code Generation Capabilities of Large Language Models
von: Padwal, Vedant
Veröffentlicht: (2026)

Asm2SrcEval: Evaluating Large Language Models for Assembly-to-Source Code Translation
von: Hamedi, Parisa, et al.
Veröffentlicht: (2025)

Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability
von: He, Mengliang, et al.
Veröffentlicht: (2025)

VulnLLMEval: A Framework for Evaluating Large Language Models in Software Vulnerability Detection and Patching
von: Zibaeirad, Arastoo, et al.
Veröffentlicht: (2024)

RobuNFR: Evaluating the Robustness of Large Language Models on Non-Functional Requirements Aware Code Generation
von: Lin, Feng, et al.
Veröffentlicht: (2025)

Evaluating Large Language Models for Functional and Maintainable Code in Industrial Settings: A Case Study at ASML
von: Mundhra, Yash, et al.
Veröffentlicht: (2025)

CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluation
von: Yin, Wenjing, et al.
Veröffentlicht: (2025)

Generating Energy-Efficient Code via Large-Language Models -- Where are we now?
von: Apsan, Radu, et al.
Veröffentlicht: (2025)

HFuzzer: Testing Large Language Models for Package Hallucinations via Phrase-based Fuzzing
von: Zhao, Yukai, et al.
Veröffentlicht: (2025)

Effective Code Membership Inference for Code Completion Models via Adversarial Prompts
von: Jiang, Yuan, et al.
Veröffentlicht: (2025)

Towards Reliable Evaluation of Neural Program Repair with Natural Robustness Testing
von: Le-Cong, Thanh, et al.
Veröffentlicht: (2024)

TREAT: A Code LLMs Trustworthiness / Reliability Evaluation and Testing Framework
von: Gao, Shuzheng, et al.
Veröffentlicht: (2025)