Dimensionality-Aware Outlier Detection: Theoretical and Experimental Analysis
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913322893312000 |
|---|---|
| author | Anderberg, Alastair Bailey, James Campello, Ricardo J. G. B. Houle, Michael E. Marques, Henrique O. Radovanović, Miloš Zimek, Arthur |
| author_facet | Anderberg, Alastair Bailey, James Campello, Ricardo J. G. B. Houle, Michael E. Marques, Henrique O. Radovanović, Miloš Zimek, Arthur |
| contents | We present a nonparametric method for outlier detection that takes full account of local variations in intrinsic dimensionality within the dataset. Using the theory of Local Intrinsic Dimensionality (LID), our 'dimensionality-aware' outlier detection method, DAO, is derived as an estimator of an asymptotic local expected density ratio involving the query point and a close neighbor drawn at random. The dimensionality-aware behavior of DAO is due to its use of local estimation of LID values in a theoretically-justified way. Through comprehensive experimentation on more than 800 synthetic and real datasets, we show that DAO significantly outperforms three popular and important benchmark outlier detection methods: Local Outlier Factor (LOF), Simplified LOF, and kNN. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_05453 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Dimensionality-Aware Outlier Detection: Theoretical and Experimental Analysis Anderberg, Alastair Bailey, James Campello, Ricardo J. G. B. Houle, Michael E. Marques, Henrique O. Radovanović, Miloš Zimek, Arthur Machine Learning Artificial Intelligence 68T99 (Primary) 62G07, 62G32, 62H30 (Secondary) We present a nonparametric method for outlier detection that takes full account of local variations in intrinsic dimensionality within the dataset. Using the theory of Local Intrinsic Dimensionality (LID), our 'dimensionality-aware' outlier detection method, DAO, is derived as an estimator of an asymptotic local expected density ratio involving the query point and a close neighbor drawn at random. The dimensionality-aware behavior of DAO is due to its use of local estimation of LID values in a theoretically-justified way. Through comprehensive experimentation on more than 800 synthetic and real datasets, we show that DAO significantly outperforms three popular and important benchmark outlier detection methods: Local Outlier Factor (LOF), Simplified LOF, and kNN. |
| title | Dimensionality-Aware Outlier Detection: Theoretical and Experimental Analysis |
| topic | Machine Learning Artificial Intelligence 68T99 (Primary) 62G07, 62G32, 62H30 (Secondary) |
| url | https://arxiv.org/abs/2401.05453 |