Cover Image

Challenges in Testing Large Language Model Based Software: A Faceted Taxonomy

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dobslaw, Felix, Feldt, Robert, Yoon, Juyeon, Yoo, Shin
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Adaptive Testing for LLM-Based Applications: A Diversity-based Approach
by: Yoon, Juyeon, et al.
Published: (2025)

Understanding on the Edge: LLM-generated Boundary Test Explanations
by: Akbarova, Sabinakhon, et al.
Published: (2026)

Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputs
by: Yoon, Juyeon, et al.
Published: (2025)

The Expert Validation Framework (EVF): Enabling Domain Expert Control in AI Engineering
by: Gren, Lucas, et al.
Published: (2026)

Automating a Complete Software Test Process Using LLMs: An Automotive Case Study
by: Wang, Shuai, et al.
Published: (2025)

The Semi-Executable Stack: Agentic Software Engineering and the Expanding Scope of SE
by: Feldt, Robert, et al.
Published: (2026)

Large Language Models for Software Testing: A Research Roadmap
by: Augusto, Cristian, et al.
Published: (2025)

SETBVE: Quality-Diversity Driven Exploration of Software Boundary Behaviors
by: Akbarova, Sabinakhon, et al.
Published: (2025)

Software Engineering for Large Language Models: Research Status, Challenges and the Road Ahead
by: Rao, Hongzhou, et al.
Published: (2025)

Large Language Model-Based Agents for Software Engineering: A Survey
by: Liu, Junwei, et al.
Published: (2024)

DomAgent: Leveraging Knowledge Graphs and Case-Based Reasoning for Domain-Specific Code Generation
by: Wang, Shuai, et al.
Published: (2026)

Reasoning-Based Software Testing
by: Giamattei, Luca, et al.
Published: (2023)

LangBiTe: A Platform for Testing Bias in Large Language Models
by: Morales, Sergio, et al.
Published: (2024)

Explaining Software Vulnerabilities with Large Language Models
by: Johnson, Oshando, et al.
Published: (2025)

Harden and Catch for Just-in-Time Assured LLM-Based Software Testing: Open Research Challenges
by: Harman, Mark, et al.
Published: (2025)

ArchCode: Incorporating Software Requirements in Code Generation with Large Language Models
by: Han, Hojae, et al.
Published: (2024)

Comparing Open-Source and Commercial LLMs for Domain-Specific Analysis and Reporting: Software Engineering Challenges and Design Trade-offs
by: Koraag, Theo, et al.
Published: (2025)

Agentic Software Issue Resolution with Large Language Models: A Survey
by: Jiang, Zhonghao, et al.
Published: (2025)

Large Language Models for Software Engineering: A Systematic Literature Review
by: Hou, Xinyi, et al.
Published: (2023)

CSA-Trans: Code Structure Aware Transformer for AST
by: Oh, Saeyoon, et al.
Published: (2024)

A Systematic Literature Review on Detecting Software Vulnerabilities with Large Language Models
by: Kaniewski, Sabrina, et al.
Published: (2025)

Metamorphic Testing of Large Language Models for Natural Language Processing
by: Cho, Steven, et al.
Published: (2025)

Redefining Developer Assistance: Through Large Language Models in Software Ecosystem
by: Banerjee, Somnath, et al.
Published: (2023)

On the Challenges of Fuzzing Techniques via Large Language Models
by: Huang, Linghan, et al.
Published: (2024)

Hallucination by Code Generation LLMs: Taxonomy, Benchmarks, Mitigation, and Challenges
by: Lee, Yunseo, et al.
Published: (2025)

LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities
by: Tang, Yongjian, et al.
Published: (2026)

A Tertiary Review of Large Language Model-Based Code Generating Tasks: Trends, Challenges, and Future Directions
by: Chochlov, Muslim, et al.
Published: (2026)

Evaluating LLM-Based Test Generation Under Software Evolution
by: Haroon, Sabaat, et al.
Published: (2026)

A Tool for Generating Exceptional Behavior Tests With Large Language Models
by: Zhong, Linghan, et al.
Published: (2025)

Leveraging Large Language Models for Code Translation and Software Development in Scientific Computing
by: Dhruv, Akash, et al.
Published: (2024)

Automatic techniques for issue report classification: A systematic mapping study
by: Laiq, Muhammad, et al.
Published: (2025)

VulnLLMEval: A Framework for Evaluating Large Language Models in Software Vulnerability Detection and Patching
by: Zibaeirad, Arastoo, et al.
Published: (2024)

A Taxonomy of Foundation Model based Systems through the Lens of Software Architecture
by: Lu, Qinghua, et al.
Published: (2023)

Automating Structural Analysis Across Multiple Software Platforms Using Large Language Models
by: Geng, Ziheng, et al.
Published: (2026)

CIAO - Code In Architecture Out - Automated Software Architecture Documentation with Large Language Models
by: De Luca, Marco, et al.
Published: (2026)

LoCoBench: A Benchmark for Long-Context Large Language Models in Complex Software Engineering
by: Qiu, Jielin, et al.
Published: (2025)

Automated Non-Functional Requirements Generation in Software Engineering with Large Language Models: A Comparative Study
by: Almonte, Jomar Thomas, et al.
Published: (2025)

Software Development Life Cycle Perspective: A Survey of Benchmarks for Code Large Language Models and Agents
by: Wang, Kaixin, et al.
Published: (2025)

Querying Large Automotive Software Models: Agentic vs. Direct LLM Approaches
by: Mazur, Lukasz, et al.
Published: (2025)

MaxMind: A Memory Loop Network to Enhance Software Productivity based on Large Language Models
by: Dong, Yuchen, et al.
Published: (2024)