Cover Image

Reliability of Large Language Models for Design Synthesis: An Empirical Study of Variance, Prompt Sensitivity, and Method Scaffolding

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Iftikhar, Rabia, Rausch, Andreas
Format:	Preprint
Published:	2026
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Benchmarking Design Pattern Detection Under Obfuscation: Reproducing and Evaluating Attention-Based Detection Method
by: Shenoy, Manthan, et al.
Published: (2025)

Guidelines to Prompt Large Language Models for Code Generation: An Empirical Characterization
by: Midolo, Alessandro, et al.
Published: (2026)

Evaluating Large Language Models for Code Translation: Effects of Prompt Language and Prompt Design
by: Aljagthami, Aamer, et al.
Published: (2025)

Bias Ahead: Sensitive Prompts as Early Warnings for Fairness in Large Language Models
by: Voria, Gianmario, et al.
Published: (2026)

Generating Software Architecture Description from Source Code using Reverse Engineering and Large Language Model
by: Hatahet, Ahmad, et al.
Published: (2025)

A Pilot Study on Detecting Software Design Patterns with Large Language Models: An Empirical Evaluation
by: Chowdhury, Oishik, et al.
Published: (2026)

Large Language Models for Fault Localization: An Empirical Study
by: Xiao, YingJian, et al.
Published: (2025)

LLM-Based Design Pattern Detection
by: Schindler, Christian, et al.
Published: (2025)

VeriODD: From YAML to SMT-LIB -- Automating Verification of Operational Design Domains
by: Rafie, Bassel, et al.
Published: (2025)

Crash Report Enhancement with Large Language Models: An Empirical Study
by: Fahim, S M Farah Al, et al.
Published: (2025)

An Empirical Study on the Code Refactoring Capability of Large Language Models
by: Cordeiro, Jonathan, et al.
Published: (2024)

An Empirical Study of Sustainability in Prompt-driven Test Script Generation Using Small Language Models
by: Kumari, Pragati, et al.
Published: (2026)

An Empirical Study of Python Library Migration Using Large Language Models
by: Islam, Md Mohayeminul, et al.
Published: (2025)

Guidelines for Empirical Studies in Software Engineering involving Large Language Models
by: Baltes, Sebastian, et al.
Published: (2025)

Prompting in the Wild: An Empirical Study of Prompt Evolution in Software Repositories
by: Tafreshipour, Mahan, et al.
Published: (2024)

Large Language Models for Automated Web-Form-Test Generation: An Empirical Study
by: Li, Tao, et al.
Published: (2024)

Automated Commit Message Generation with Large Language Models: An Empirical Study and Beyond
by: Xue, Pengyu, et al.
Published: (2024)

Large Language Models for Mobile GUI Text Input Generation: An Empirical Study
by: Cui, Chenhui, et al.
Published: (2024)

A Large-scale Empirical Study on Fine-tuning Large Language Models for Unit Testing
by: Shang, Ye, et al.
Published: (2024)

Behavioral Augmentation of UML Class Diagrams: An Empirical Study of Large Language Models for Method Generation
by: Rouabhia, Djaber, et al.
Published: (2025)

Empirical Studies of Parameter Efficient Methods for Large Language Models of Code and Knowledge Transfer to R
by: Esmaeili, Amirreza, et al.
Published: (2024)

An Empirical Study on the Effects of System Prompts in Instruction-Tuned Models for Code Generation
by: Cheng, Zaiyu, et al.
Published: (2026)

An Empirical Study on Influence-Based Pretraining Data Selection for Code Large Language Models
by: Xing, Chengli, et al.
Published: (2026)

Evaluating Reliability Gaps in Large Language Model Safety via Repeated Prompt Sampling
by: Broadwater, Keita
Published: (2026)

Analyzing Prompt Influence on Automated Method Generation: An Empirical Study with Copilot
by: Fagadau, Ionut Daniel, et al.
Published: (2024)

Prediction Model of Motivators and Demotivators of Integrating Large Language Models in Software Engineering Education: An Empirical Study
by: Khan, Maryam, et al.
Published: (2026)

An Empirical Study of Large Language Models for Type and Call Graph Analysis in Python and JavaScript
by: Venkatesh, Ashwin Prasad Shivarpatna, et al.
Published: (2024)

An Empirical Study on Low-Code Programming using Traditional vs Large Language Model Support
by: Liu, Yongkun, et al.
Published: (2024)

When Large Language Models Meet UAV Projects: An Empirical Study from Developers' Perspective
by: Chen, Yihua, et al.
Published: (2025)

Bugs in Large Language Models Generated Code: An Empirical Study
by: Tambon, Florian, et al.
Published: (2024)

Empirical Evaluation of Large Language Models in Automated Program Repair
by: Sun, Jiajun, et al.
Published: (2025)

Failure-Aware Enhancements for Large Language Model (LLM) Code Generation: An Empirical Study on Decision Framework
by: Shen, Jianru, et al.
Published: (2026)

An Exploratory Study of Bayesian Prompt Optimization for Test-Driven Code Generation with Large Language Models
by: Tomar, Shlok, et al.
Published: (2025)

Development of Automated Software Design Document Review Methods Using Large Language Models
by: Fukuda, Takasaburo, et al.
Published: (2025)

LLMs-Powered Real-Time Fault Injection: An Approach Toward Intelligent Fault Test Cases Generation
by: Abboush, Mohammad, et al.
Published: (2025)

Ensembling Large Language Models for Code Vulnerability Detection: An Empirical Evaluation
by: Sun, Zhihong, et al.
Published: (2025)

Using Large Language Models to Generate JUnit Tests: An Empirical Study
by: Siddiq, Mohammed Latif, et al.
Published: (2023)

An Empirical Study on the Effectiveness of Large Language Models for Binary Code Understanding
by: Shang, Xiuwei, et al.
Published: (2025)

An Empirical Study on Capability of Large Language Models in Understanding Code Semantics
by: Nguyen, Thu-Trang, et al.
Published: (2024)

Are We SOLID Yet? An Empirical Study on Prompting LLMs to Detect Design Principle Violations
by: Pehlivan, Fatih, et al.
Published: (2025)