Cover Image

230,439 Test Failures Later: An Empirical Evaluation of Flaky Failure Classifiers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Alshammari, Abdulrahman, Ammann, Paul, Hilton, Michael, Bell, Jonathan
Format:	Preprint
Published:	2024
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Systemic Flakiness: An Empirical Analysis of Co-Occurring Flaky Test Failures
by: Parry, Owain, et al.
Published: (2025)

A Dataset of Reproducible Flaky-Test Failures
by: Rafi, Suzzana, et al.
Published: (2026)

On the Diagnosis of Flaky Job Failures: Understanding and Prioritizing Failure Categories
by: Aïdasso, Henri, et al.
Published: (2025)

Can We Classify Flaky Tests Using Only Test Code? An LLM-Based Empirical Study
by: Berndt, Alexander, et al.
Published: (2026)

The Effects of Computational Resources on Flaky Tests
by: Silva, Denini, et al.
Published: (2023)

FlakeRanker: Automated Identification and Prioritization of Flaky Job Failure Categories
by: Aïdasso, Henri
Published: (2025)

A Practical Framework for Flaky Failure Triage in Distributed Database Continuous Integration
by: Zhu, Jun-Peng, et al.
Published: (2026)

Do Test and Environmental Complexity Increase Flakiness? An Empirical Study of SAP HANA
by: Berndt, Alexander, et al.
Published: (2024)

A Systematic Evaluation of Environmental Flakiness in JavaScript Tests
by: Hashemi, Negar, et al.
Published: (2026)

Detecting and Evaluating Order-Dependent Flaky Tests in JavaScript
by: Hashemi, Negar, et al.
Published: (2025)

Taming Timeout Flakiness: An Empirical Study of SAP HANA
by: Berndt, Alexander, et al.
Published: (2024)

The Vocabulary of Flaky Tests in the Context of SAP HANA
by: Berndt, Alexander, et al.
Published: (2026)

Automated Test Validators for Flaky Cyber-Physical System Simulators: Approach and Evaluation
by: Jodat, Baharin A., et al.
Published: (2025)

Is this Build Failure Related to my Patch? An Empirical Study of Unrelated Build Failures in Continuous Integration
by: Huang, Andie, et al.
Published: (2026)

Flaky Tests in a Large Industrial Database Management System: An Empirical Study of Fixed Issue Reports for SAP HANA
by: Berndt, Alexander, et al.
Published: (2026)

JS-TOD: Detecting Order-Dependent Flaky Tests in Jest
by: Hashemi, Negar, et al.
Published: (2025)

Detecting Flaky Tests in Quantum Software: A Dynamic Approach
by: Kim, Dongchan, et al.
Published: (2025)

Reduction of Test Re-runs by Prioritizing Potential Order Dependent Flaky Tests
by: Iqbal, Hasnain, et al.
Published: (2025)

A Generic Approach to Fix Test Flakiness in Real-World Projects
by: Chen, Yang, et al.
Published: (2024)

Dockerfile Flakiness: Characterization and Repair
by: Shabani, Taha, et al.
Published: (2024)

On the Flakiness of LLM-Generated Tests for Industrial and Open-Source Database Management Systems
by: Berndt, Alexander, et al.
Published: (2026)

NeuroFlake: A Neuro-Symbolic LLM Framework for Flaky Test Classification
by: Hoque, Khondaker Tasnia, et al.
Published: (2026)

A Preliminary Study of Fixed Flaky Tests in Rust Projects on GitHub
by: Schroeder, Tom, et al.
Published: (2025)

FlakyGuard: Automatically Fixing Flaky Tests at Industry Scale
by: Li, Chengpeng, et al.
Published: (2025)

Dissecting Bug Triggers and Failure Modes in Modern Agentic Frameworks: An Empirical Study
by: Zhang, Xiaowen, et al.
Published: (2026)

On the Illusion of Success: An Empirical Study of Build Reruns and Silent Failures in Industrial CI
by: Aïdasso, Henri, et al.
Published: (2025)

FlakyFix: Using Large Language Models for Predicting Flaky Test Fix Categories and Test Code Repair
by: Fatima, Sakina, et al.
Published: (2023)

WEFix: Intelligent Automatic Generation of Explicit Waits for Efficient Web End-to-End Flaky Tests
by: Liu, Xinyue, et al.
Published: (2024)

Detecting and Mitigating Flakiness in REST API Fuzzing
by: Zhang, Man, et al.
Published: (2026)

Identifying Flaky Tests in Quantum Code: A Machine Learning Approach
by: Kaur, Khushdeep, et al.
Published: (2025)

Easy over Hard: A Simple Baseline for Test Failures Causes Prediction
by: Gao, Zhipeng, et al.
Published: (2024)

An Empirical Study on Failures in Automated Issue Solving
by: Liu, Simiao, et al.
Published: (2025)

Failure-Aware Enhancements for Large Language Model (LLM) Code Generation: An Empirical Study on Decision Framework
by: Shen, Jianru, et al.
Published: (2026)

Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories
by: Majgaonkar, Oorja, et al.
Published: (2025)

Learning From Lessons Learned: Preliminary Findings From a Study of Learning From Failure
by: Sillito, Jonathan, et al.
Published: (2024)

Systematic Evaluation of Deep Learning Models for Log-based Failure Prediction
by: Hadadi, Fatemeh, et al.
Published: (2023)

LLMorpheus: Mutation Testing using Large Language Models
by: Tip, Frank, et al.
Published: (2024)

FlaKat: A Machine Learning-Based Categorization Framework for Flaky Tests
by: Lin, Shizhe, et al.
Published: (2024)

LLM-Based Automated Diagnosis Of Integration Test Failures At Google
by: Ziftci, Celal, et al.
Published: (2026)

A Large Language Model Approach to Identify Flakiness in C++ Projects
by: Sun, Xin, et al.
Published: (2024)