Cover Image

A Software Engineering Perspective on Testing Large Language Models: Research, Practice, Tools and Benchmarks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hudson, Sinclair, Jit, Sophia, Hu, Boyue Caroline, Chechik, Marsha
Format:	Preprint
Published:	2024
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Future of Software Engineering Research: The SIGSOFT Perspective
by: Di Penta, Massimiliano, et al.
Published: (2026)

PLACIDUS: Engineering Product Lines of Rigorous Assurance Cases
by: Murphy, Logan, et al.
Published: (2024)

Diagnosis via Proofs of Unsatisfiability for First-Order Logic with Relational Objects
by: Feng, Nick, et al.
Published: (2024)

Assurance Case Development for Evolving Software Product Lines: A Formal Approach
by: Murphy, Logan, et al.
Published: (2025)

Abductive Vibe Coding (Extended Abstract)
by: Murphy, Logan, et al.
Published: (2026)

Testing Research Software: An In-Depth Survey of Practices, Methods, and Tools
by: Eisty, Nasir U., et al.
Published: (2025)

Operationalizing Human Values in the Requirements Engineering Process of Ethics-Aware Autonomous Systems
by: Júnior, Everaldo Silva, et al.
Published: (2026)

Assessing and Advancing Benchmarks for Evaluating Large Language Models in Software Engineering Tasks
by: Hu, Xing, et al.
Published: (2025)

Abstraction Engineering
by: Bencomo, Nelly, et al.
Published: (2024)

Practical Feasibility of Sustainable Software Engineering Tools and Techniques
by: Ghanta, Satwik, et al.
Published: (2026)

Developers' Perspectives on Software Licensing: Current Practices, Challenges, and Tools
by: Wintersgill, Nathan, et al.
Published: (2025)

The Current Challenges of Software Engineering in the Era of Large Language Models
by: Gao, Cuiyun, et al.
Published: (2024)

Normative Requirements Operationalization with Large Language Models
by: Feng, Nick, et al.
Published: (2024)

A Survey on Large Language Models for Software Engineering
by: Zhang, Quanjun, et al.
Published: (2023)

Peer Code Review in Research Software Development: The Research Software Engineer Perspective
by: Malik, Md Ariful Islam, et al.
Published: (2025)

Do Research Software Engineers and Software Engineering Researchers Speak the Same Language?
by: Kehrer, Timo, et al.
Published: (2025)

Large Language Models for Software Testing: A Research Roadmap
by: Augusto, Cristian, et al.
Published: (2025)

Automated Testing of the GUI of a Real-Life Engineering Software using Large Language Models
by: Rosenbach, Tim, et al.
Published: (2025)

SELU: A Software Engineering Language Understanding Benchmark
by: Peña, Fabian C., et al.
Published: (2025)

Large Language Models in Thematic Analysis: Prompt Engineering, Evaluation, and Guidelines for Qualitative Software Engineering Research
by: Montes, Cristina Martinez, et al.
Published: (2025)

Exploring the Power of Diffusion Large Language Models for Software Engineering: An Empirical Investigation
by: Zhang, Jingyao, et al.
Published: (2025)

Empirical and Sustainability Aspects of Software Engineering Research in the Era of Large Language Models: A Reflection
by: Williams, David, et al.
Published: (2025)

Triage in Software Engineering: A Systematic Review of Research and Practice
by: Zhao, Yongxin, et al.
Published: (2025)

Evaluation of LLM-Based Software Engineering Tools: Practices, Challenges, and Future Directions
by: Torun, Utku Boran, et al.
Published: (2026)

Action Research with Industrial Software Engineering -- An Educational Perspective
by: Dittrich, Yvonne, et al.
Published: (2024)

From Literature to Practice: Exploring Fairness Testing Tools for the Software Industry Adoption
by: Nguyen, Thanh, et al.
Published: (2024)

Perspective of Software Engineering Researchers on Machine Learning Practices Regarding Research, Review, and Education
by: Mojica-Hanke, Anamaria, et al.
Published: (2024)

Benchmarking Large Language Models for Multi-Language Software Vulnerability Detection
by: Zhang, Ting, et al.
Published: (2025)

A Vision on Open Science for the Evolution of Software Engineering Research and Practice
by: OliveiraJr, Edson, et al.
Published: (2024)

Multi-CoLoR: Context-Aware Localization and Reasoning across Multi-Language Codebases
by: Vats, Indira, et al.
Published: (2026)

Augmenting the Generality and Performance of Large Language Models for Software Engineering
by: Peña, Fabian C.
Published: (2025)

Towards an Understanding of Large Language Models in Software Engineering Tasks
by: Zheng, Zibin, et al.
Published: (2023)

Regression Testing in Remote and Hybrid Software Teams: An Exploratory Study of Processes, Tools, and Practices
by: Pascoal, Juliane, et al.
Published: (2026)

Software Testing with Large Language Models: An Interview Study with Practitioners
by: Santana, Maria Deolinda, et al.
Published: (2025)

Software Testing with Large Language Models: Survey, Landscape, and Vision
by: Wang, Junjie, et al.
Published: (2023)

Large Language Models for Software Testing Education: an Experience Report
by: Yang, Peng, et al.
Published: (2026)

Software Fairness Testing in Practice
by: Santos, Ronnie de Souza, et al.
Published: (2025)

Revisiting Sentiment Analysis for Software Engineering in the Era of Large Language Models
by: Zhang, Ting, et al.
Published: (2023)

Guidelines for Empirical Studies in Software Engineering involving Large Language Models
by: Baltes, Sebastian, et al.
Published: (2025)

Attributes to Support the Formulation of Practically Relevant Research Problems in Software Engineering
by: Pereira, Anrafel Fernandes, et al.
Published: (2025)