RCAEval: A Benchmark for Root Cause Analysis of Microservice Systems with Telemetry Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912216348884992 |
|---|---|
| author | Pham, Luan Zhang, Hongyu Ha, Huong Salim, Flora Zhang, Xiuzhen |
| author_facet | Pham, Luan Zhang, Hongyu Ha, Huong Salim, Flora Zhang, Xiuzhen |
| contents | Root cause analysis (RCA) for microservice systems has gained significant attention in recent years. However, there is still no standard benchmark that includes large-scale datasets and supports comprehensive evaluation environments. In this paper, we introduce RCAEval, an open-source benchmark that provides datasets and an evaluation environment for RCA in microservice systems. First, we introduce three comprehensive datasets comprising 735 failure cases collected from three microservice systems, covering various fault types observed in real-world failures. Second, we present a comprehensive evaluation framework that includes fifteen reproducible baselines covering a wide range of RCA approaches, with the ability to evaluate both coarse-grained and fine-grained RCA. We hope that this ready-to-use benchmark will enable researchers and practitioners to conduct extensive analysis and pave the way for robust new solutions for RCA of microservice systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_17015 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RCAEval: A Benchmark for Root Cause Analysis of Microservice Systems with Telemetry Data Pham, Luan Zhang, Hongyu Ha, Huong Salim, Flora Zhang, Xiuzhen Software Engineering Root cause analysis (RCA) for microservice systems has gained significant attention in recent years. However, there is still no standard benchmark that includes large-scale datasets and supports comprehensive evaluation environments. In this paper, we introduce RCAEval, an open-source benchmark that provides datasets and an evaluation environment for RCA in microservice systems. First, we introduce three comprehensive datasets comprising 735 failure cases collected from three microservice systems, covering various fault types observed in real-world failures. Second, we present a comprehensive evaluation framework that includes fifteen reproducible baselines covering a wide range of RCA approaches, with the ability to evaluate both coarse-grained and fine-grained RCA. We hope that this ready-to-use benchmark will enable researchers and practitioners to conduct extensive analysis and pave the way for robust new solutions for RCA of microservice systems. |
| title | RCAEval: A Benchmark for Root Cause Analysis of Microservice Systems with Telemetry Data |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2412.17015 |