Quantifying Statistical Significance of Deep Nearest Neighbor Anomaly Detection via Selective Inference
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918220180488192 |
|---|---|
| author | Niihori, Mizuki Nishino, Shuichi Katsuoka, Teruyuki Shiraishi, Tomohiro Taji, Kouichi Takeuchi, Ichiro |
| author_facet | Niihori, Mizuki Nishino, Shuichi Katsuoka, Teruyuki Shiraishi, Tomohiro Taji, Kouichi Takeuchi, Ichiro |
| contents | In real-world applications, anomaly detection (AD) often operates without access to anomalous data, necessitating semi-supervised methods that rely solely on normal data. Among these methods, deep k-nearest neighbor (deep kNN) AD stands out for its interpretability and flexibility, leveraging distance-based scoring in deep latent spaces.Despite its strong performance, deep kNN lacks a mechanism to quantify uncertainty-an essential feature for critical applications such as industrial inspection. To address this limitation, we propose a statistical framework that quantifies the significance of detected anomalies in the form of p-values, thereby enabling control over false positive rates at a user-specified significance level (e.g.,0.05). A central challenge lies in managing selection bias, which we tackle using Selective Inference-a principled method for conducting inference conditioned on data-driven selections. We evaluate our method on diverse datasets and demonstrate that it provides reliable AD well-suited for industrial use cases. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_12978 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Quantifying Statistical Significance of Deep Nearest Neighbor Anomaly Detection via Selective Inference Niihori, Mizuki Nishino, Shuichi Katsuoka, Teruyuki Shiraishi, Tomohiro Taji, Kouichi Takeuchi, Ichiro Machine Learning In real-world applications, anomaly detection (AD) often operates without access to anomalous data, necessitating semi-supervised methods that rely solely on normal data. Among these methods, deep k-nearest neighbor (deep kNN) AD stands out for its interpretability and flexibility, leveraging distance-based scoring in deep latent spaces.Despite its strong performance, deep kNN lacks a mechanism to quantify uncertainty-an essential feature for critical applications such as industrial inspection. To address this limitation, we propose a statistical framework that quantifies the significance of detected anomalies in the form of p-values, thereby enabling control over false positive rates at a user-specified significance level (e.g.,0.05). A central challenge lies in managing selection bias, which we tackle using Selective Inference-a principled method for conducting inference conditioned on data-driven selections. We evaluate our method on diverse datasets and demonstrate that it provides reliable AD well-suited for industrial use cases. |
| title | Quantifying Statistical Significance of Deep Nearest Neighbor Anomaly Detection via Selective Inference |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2502.12978 |