Towards Unbiased Evaluation of Time-series Anomaly Detector
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909320462991360 |
|---|---|
| author | Bhattacharya, Debarpan Mukherjee, Sumanta Kamanchi, Chandramouli Ekambaram, Vijay Jati, Arindam Dayama, Pankaj |
| author_facet | Bhattacharya, Debarpan Mukherjee, Sumanta Kamanchi, Chandramouli Ekambaram, Vijay Jati, Arindam Dayama, Pankaj |
| contents | Time series anomaly detection (TSAD) is an evolving area of research motivated by its critical applications, such as detecting seismic activity, sensor failures in industrial plants, predicting crashes in the stock market, and so on. Across domains, anomalies occur significantly less frequently than normal data, making the F1-score the most commonly adopted metric for anomaly detection. However, in the case of time series, it is not straightforward to use standard F1-score because of the dissociation between `time points' and `time events'. To accommodate this, anomaly predictions are adjusted, called as point adjustment (PA), before the $F_1$-score evaluation. However, these adjustments are heuristics-based, and biased towards true positive detection, resulting in over-estimated detector performance. In this work, we propose an alternative adjustment protocol called ``Balanced point adjustment'' (BA). It addresses the limitations of existing point adjustment methods and provides guarantees of fairness backed by axiomatic definitions of TSAD evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_13053 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Unbiased Evaluation of Time-series Anomaly Detector Bhattacharya, Debarpan Mukherjee, Sumanta Kamanchi, Chandramouli Ekambaram, Vijay Jati, Arindam Dayama, Pankaj Machine Learning Applications Time series anomaly detection (TSAD) is an evolving area of research motivated by its critical applications, such as detecting seismic activity, sensor failures in industrial plants, predicting crashes in the stock market, and so on. Across domains, anomalies occur significantly less frequently than normal data, making the F1-score the most commonly adopted metric for anomaly detection. However, in the case of time series, it is not straightforward to use standard F1-score because of the dissociation between `time points' and `time events'. To accommodate this, anomaly predictions are adjusted, called as point adjustment (PA), before the $F_1$-score evaluation. However, these adjustments are heuristics-based, and biased towards true positive detection, resulting in over-estimated detector performance. In this work, we propose an alternative adjustment protocol called ``Balanced point adjustment'' (BA). It addresses the limitations of existing point adjustment methods and provides guarantees of fairness backed by axiomatic definitions of TSAD evaluation. |
| title | Towards Unbiased Evaluation of Time-series Anomaly Detector |
| topic | Machine Learning Applications |
| url | https://arxiv.org/abs/2409.13053 |