Adaptive Bernstein Change Detector for High-Dimensional Data Streams
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917566936514560 |
|---|---|
| author | Heyden, Marco Fouché, Edouard Arzamasov, Vadim Fenn, Tanja Kalinke, Florian Böhm, Klemens |
| author_facet | Heyden, Marco Fouché, Edouard Arzamasov, Vadim Fenn, Tanja Kalinke, Florian Böhm, Klemens |
| contents | Change detection is of fundamental importance when analyzing data streams. Detecting changes both quickly and accurately enables monitoring and prediction systems to react, e.g., by issuing an alarm or by updating a learning algorithm. However, detecting changes is challenging when observations are high-dimensional. In high-dimensional data, change detectors should not only be able to identify when changes happen, but also in which subspace they occur. Ideally, one should also quantify how severe they are. Our approach, ABCD, has these properties. ABCD learns an encoder-decoder model and monitors its accuracy over a window of adaptive size. ABCD derives a change score based on Bernstein's inequality to detect deviations in terms of accuracy, which indicate changes. Our experiments demonstrate that ABCD outperforms its best competitor by up to 20% in F1-score on average. It can also accurately estimate changes' subspace, together with a severity measure that correlates with the ground truth. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_12974 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Adaptive Bernstein Change Detector for High-Dimensional Data Streams Heyden, Marco Fouché, Edouard Arzamasov, Vadim Fenn, Tanja Kalinke, Florian Böhm, Klemens Machine Learning 68T05 I.2.6 Change detection is of fundamental importance when analyzing data streams. Detecting changes both quickly and accurately enables monitoring and prediction systems to react, e.g., by issuing an alarm or by updating a learning algorithm. However, detecting changes is challenging when observations are high-dimensional. In high-dimensional data, change detectors should not only be able to identify when changes happen, but also in which subspace they occur. Ideally, one should also quantify how severe they are. Our approach, ABCD, has these properties. ABCD learns an encoder-decoder model and monitors its accuracy over a window of adaptive size. ABCD derives a change score based on Bernstein's inequality to detect deviations in terms of accuracy, which indicate changes. Our experiments demonstrate that ABCD outperforms its best competitor by up to 20% in F1-score on average. It can also accurately estimate changes' subspace, together with a severity measure that correlates with the ground truth. |
| title | Adaptive Bernstein Change Detector for High-Dimensional Data Streams |
| topic | Machine Learning 68T05 I.2.6 |
| url | https://arxiv.org/abs/2306.12974 |