MechDetect: Detecting Data-Dependent Errors
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912748094357504 |
|---|---|
| author | Jung, Philipp Chandler, Nicholas Jäger, Sebastian Biessmann, Felix |
| author_facet | Jung, Philipp Chandler, Nicholas Jäger, Sebastian Biessmann, Felix |
| contents | Data quality monitoring is a core challenge in modern information processing systems. While many approaches to detect data errors or shifts have been proposed, few studies investigate the mechanisms governing error generation. We argue that knowing how errors were generated can be key to tracing and fixing them. In this study, we build on existing work in the statistics literature on missing values and propose MechDetect, a simple algorithm to investigate error generation mechanisms. Given a tabular data set and a corresponding error mask, the algorithm estimates whether or not the errors depend on the data using machine learning models. Our work extends established approaches to detect mechanisms underlying missing values and can be readily applied to other error types, provided that an error mask is available. We demonstrate the effectiveness of MechDetect in experiments on established benchmark datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_04138 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MechDetect: Detecting Data-Dependent Errors Jung, Philipp Chandler, Nicholas Jäger, Sebastian Biessmann, Felix Machine Learning Databases Information Retrieval Data quality monitoring is a core challenge in modern information processing systems. While many approaches to detect data errors or shifts have been proposed, few studies investigate the mechanisms governing error generation. We argue that knowing how errors were generated can be key to tracing and fixing them. In this study, we build on existing work in the statistics literature on missing values and propose MechDetect, a simple algorithm to investigate error generation mechanisms. Given a tabular data set and a corresponding error mask, the algorithm estimates whether or not the errors depend on the data using machine learning models. Our work extends established approaches to detect mechanisms underlying missing values and can be readily applied to other error types, provided that an error mask is available. We demonstrate the effectiveness of MechDetect in experiments on established benchmark datasets. |
| title | MechDetect: Detecting Data-Dependent Errors |
| topic | Machine Learning Databases Information Retrieval |
| url | https://arxiv.org/abs/2512.04138 |