Walk the Talk: Is Your Log-based Software Reliability Maintenance System Really Reliable?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916976120561664 |
|---|---|
| author | He, Minghua Jia, Tong Duan, Chiming Xiao, Pei Zhang, Lingzhe Wang, Kangjin Wu, Yifan Li, Ying Huang, Gang |
| author_facet | He, Minghua Jia, Tong Duan, Chiming Xiao, Pei Zhang, Lingzhe Wang, Kangjin Wu, Yifan Li, Ying Huang, Gang |
| contents | Log-based software reliability maintenance systems are crucial for sustaining stable customer experience. However, existing deep learning-based methods represent a black box for service providers, making it impossible for providers to understand how these methods detect anomalies, thereby hindering trust and deployment in real production environments. To address this issue, this paper defines a trustworthiness metric, diagnostic faithfulness, for models to gain service providers' trust, based on surveys of SREs at a major cloud provider. We design two evaluation tasks: attention-based root cause localization and event perturbation. Empirical studies demonstrate that existing methods perform poorly in diagnostic faithfulness. Consequently, we propose FaithLog, a faithful log-based anomaly detection system, which achieves faithfulness through a carefully designed causality-guided attention mechanism and adversarial consistency learning. Evaluation results on two public datasets and one industrial dataset demonstrate that the proposed method achieves state-of-the-art performance in diagnostic faithfulness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_24352 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Walk the Talk: Is Your Log-based Software Reliability Maintenance System Really Reliable? He, Minghua Jia, Tong Duan, Chiming Xiao, Pei Zhang, Lingzhe Wang, Kangjin Wu, Yifan Li, Ying Huang, Gang Software Engineering Log-based software reliability maintenance systems are crucial for sustaining stable customer experience. However, existing deep learning-based methods represent a black box for service providers, making it impossible for providers to understand how these methods detect anomalies, thereby hindering trust and deployment in real production environments. To address this issue, this paper defines a trustworthiness metric, diagnostic faithfulness, for models to gain service providers' trust, based on surveys of SREs at a major cloud provider. We design two evaluation tasks: attention-based root cause localization and event perturbation. Empirical studies demonstrate that existing methods perform poorly in diagnostic faithfulness. Consequently, we propose FaithLog, a faithful log-based anomaly detection system, which achieves faithfulness through a carefully designed causality-guided attention mechanism and adversarial consistency learning. Evaluation results on two public datasets and one industrial dataset demonstrate that the proposed method achieves state-of-the-art performance in diagnostic faithfulness. |
| title | Walk the Talk: Is Your Log-based Software Reliability Maintenance System Really Reliable? |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2509.24352 |