Cover Image

An Empirical Study of the Non-determinism of ChatGPT in Code Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ouyang, Shuyin, Zhang, Jie M., Harman, Mark, Wang, Meng
Format:	Preprint
Published:	2023
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

WildCode: An Empirical Analysis of Code Generated by ChatGPT
by: Khanmohammadi, Kobra, et al.
Published: (2025)

An Empirical Study of ChatGPT-Related Projects and Their Issues on GitHub
by: Lin, Zheng, et al.
Published: (2024)

Self-collaboration Code Generation via ChatGPT
by: Dong, Yihong, et al.
Published: (2023)

Why Do Developers Engage with ChatGPT in Issue-Tracker? Investigating Usage and Reliance on ChatGPT-Generated Code
by: Das, Joy Krishan, et al.
Published: (2024)

Studying How Configurations Impact Code Generation in LLMs: the Case of ChatGPT
by: Donato, Benedetta, et al.
Published: (2025)

Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study
by: Liu, Yi, et al.
Published: (2023)

DevGPT: Studying Developer-ChatGPT Conversations
by: Xiao, Tao, et al.
Published: (2023)

Do Prompt Patterns Affect Code Quality? A First Empirical Assessment of ChatGPT-Generated Code
by: Della Porta, Antonio, et al.
Published: (2025)

No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT
by: Liu, Zhijie, et al.
Published: (2023)

How to Refactor this Code? An Exploratory Study on Developer-ChatGPT Refactoring Conversations
by: AlOmar, Eman Abdullah, et al.
Published: (2024)

Software Vulnerability Prediction in Low-Resource Languages: An Empirical Study of CodeBERT and ChatGPT
by: Le, Triet H. M., et al.
Published: (2024)

Exploring ChatGPT App Ecosystem: Distribution, Deployment and Security
by: Yan, Chuan, et al.
Published: (2024)

On the Possibility of Breaking Copyleft Licenses When Reusing Code Generated by ChatGPT
by: Colombo, Gaia, et al.
Published: (2025)

An Empirical Study on Developers Shared Conversations with ChatGPT in GitHub Pull Requests and Issues
by: Hao, Huizi, et al.
Published: (2024)

Knowledge-Enhanced Program Repair for Data Science Code
by: Ouyang, Shuyin, et al.
Published: (2025)

Unveiling the Role of ChatGPT in Software Development: Insights from Developer-ChatGPT Interactions on GitHub
by: Li, Ruiyin, et al.
Published: (2025)

DSCodeBench: A Realistic Benchmark for Data Science Code Generation
by: Ouyang, Shuyin, et al.
Published: (2025)

ChatGPT for Code Refactoring: Analyzing Topics, Interaction, and Effective Prompts
by: AlOmar, Eman Abdullah, et al.
Published: (2025)

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions
by: Kabir, Samia, et al.
Published: (2023)

Is ChatGPT 3 safe for students?
by: Kotovich, Julia, et al.
Published: (2024)

ChatGPT vs. DeepSeek: A Comparative Study on AI-Based Code Generation
by: Manik, Md Motaleb Hossen
Published: (2025)

You Augment Me: Exploring ChatGPT-based Data Augmentation for Semantic Code Search
by: Wang, Yanlin, et al.
Published: (2024)

SOGPTSpotter: Detecting ChatGPT-Generated Answers on Stack Overflow
by: Ma, Suyu, et al.
Published: (2026)

Evaluating and Improving ChatGPT-Based Expansion of Abbreviations
by: Jiang, Yanjie, et al.
Published: (2024)

Can ChatGPT Generate Realistic Synthetic System Requirement Specifications? Results of a Case Study
by: Mattukat, Alex R., et al.
Published: (2026)

Can ChatGPT Support Developers? An Empirical Evaluation of Large Language Models for Code Generation
by: Jin, Kailun, et al.
Published: (2024)

Investigating the Utility of ChatGPT in the Issue Tracking System: An Exploratory Study
by: Das, Joy Krishan, et al.
Published: (2024)

Evaluating ChatGPT-3.5 Efficiency in Solving Coding Problems of Different Complexity Levels: An Empirical Analysis
by: Li, Minda, et al.
Published: (2024)

No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generation
by: Yuan, Zhiqiang, et al.
Published: (2023)

Experimental Analysis of Productive Interaction Strategy with ChatGPT: User Study on Function and Project-level Code Generation Tasks
by: Hyun, Sangwon, et al.
Published: (2025)

Prompt-Enhanced Software Vulnerability Detection Using ChatGPT
by: Zhang, Chenyuan, et al.
Published: (2023)

Pros and Cons! Evaluating ChatGPT on Software Vulnerability
by: Yin, Xin
Published: (2024)

The role of library versions in Developer-ChatGPT conversations
by: Raj, Rachna, et al.
Published: (2024)

ChatGPT in Introductory Programming: Counterbalanced Evaluation of Code Quality, Conceptual Learning, and Student Perceptions
by: Andleeb, Shiza, et al.
Published: (2025)

ChatGPT as a Software Development Bot: A Project-based Study
by: Waseem, Muhammad, et al.
Published: (2023)

Beyond Execution: Static-Analysis Rewards and Hint-Conditioned Diffusion RL for Code Generation
by: Ouyang, Shuyin, et al.
Published: (2026)

ChatGPT Inaccuracy Mitigation during Technical Report Understanding: Are We There Yet?
by: Tamanna, Salma Begum, et al.
Published: (2024)

Is ChatGPT a Good Software Librarian? An Exploratory Study on the Use of ChatGPT for Software Library Recommendations
by: Latendresse, Jasmine, et al.
Published: (2024)

How I Learned to Stop Worrying and Love ChatGPT
by: Przymus, Piotr, et al.
Published: (2025)

Unveiling ChatGPT's Usage in Open Source Projects: A Mining-based Study
by: Tufano, Rosalia, et al.
Published: (2024)