Formally Exploring Time-Series Anomaly Detection Evaluation Metrics
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914104359256064 |
|---|---|
| author | Wagner, Dennis Nair, Arjun Franks, Billy Joe Arweiler, Justus Muraleedharan, Aparna Jungjohann, Indra Hartung, Fabian Ahuja, Mayank C. Balinskyy, Andriy Varshneya, Saurabh Syed, Nabeel Hussain Nagda, Mayank Liznerski, Phillip Reithermann, Steffen Rudolph, Maja Vollmer, Sebastian Schulz, Ralf Katz, Torsten Mandt, Stephan Bortz, Michael Leitte, Heike Neider, Daniel Burger, Jakob Jirasek, Fabian Hasse, Hans Fellenz, Sophie Kloft, Marius |
| author_facet | Wagner, Dennis Nair, Arjun Franks, Billy Joe Arweiler, Justus Muraleedharan, Aparna Jungjohann, Indra Hartung, Fabian Ahuja, Mayank C. Balinskyy, Andriy Varshneya, Saurabh Syed, Nabeel Hussain Nagda, Mayank Liznerski, Phillip Reithermann, Steffen Rudolph, Maja Vollmer, Sebastian Schulz, Ralf Katz, Torsten Mandt, Stephan Bortz, Michael Leitte, Heike Neider, Daniel Burger, Jakob Jirasek, Fabian Hasse, Hans Fellenz, Sophie Kloft, Marius |
| contents | Undetected anomalies in time series can trigger catastrophic failures in safety-critical systems, such as chemical plant explosions or power grid outages. Although many detection methods have been proposed, their performance remains unclear because current metrics capture only narrow aspects of the task and often yield misleading results. We address this issue by introducing verifiable properties that formalize essential requirements for evaluating time-series anomaly detection. These properties enable a theoretical framework that supports principled evaluations and reliable comparisons. Analyzing 37 widely used metrics, we show that most satisfy only a few properties, and none satisfy all, explaining persistent inconsistencies in prior results. To close this gap, we propose LARM, a flexible metric that provably satisfies all properties, and extend it to ALARM, an advanced variant meeting stricter requirements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_17562 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Formally Exploring Time-Series Anomaly Detection Evaluation Metrics Wagner, Dennis Nair, Arjun Franks, Billy Joe Arweiler, Justus Muraleedharan, Aparna Jungjohann, Indra Hartung, Fabian Ahuja, Mayank C. Balinskyy, Andriy Varshneya, Saurabh Syed, Nabeel Hussain Nagda, Mayank Liznerski, Phillip Reithermann, Steffen Rudolph, Maja Vollmer, Sebastian Schulz, Ralf Katz, Torsten Mandt, Stephan Bortz, Michael Leitte, Heike Neider, Daniel Burger, Jakob Jirasek, Fabian Hasse, Hans Fellenz, Sophie Kloft, Marius Machine Learning Undetected anomalies in time series can trigger catastrophic failures in safety-critical systems, such as chemical plant explosions or power grid outages. Although many detection methods have been proposed, their performance remains unclear because current metrics capture only narrow aspects of the task and often yield misleading results. We address this issue by introducing verifiable properties that formalize essential requirements for evaluating time-series anomaly detection. These properties enable a theoretical framework that supports principled evaluations and reliable comparisons. Analyzing 37 widely used metrics, we show that most satisfy only a few properties, and none satisfy all, explaining persistent inconsistencies in prior results. To close this gap, we propose LARM, a flexible metric that provably satisfies all properties, and extend it to ALARM, an advanced variant meeting stricter requirements. |
| title | Formally Exploring Time-Series Anomaly Detection Evaluation Metrics |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.17562 |