Unreflected Use of Tabular Data Repositories Can Undermine Research Quality
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917953509785600 |
|---|---|
| author | Tschalzev, Andrej Purucker, Lennart Lüdtke, Stefan Hutter, Frank Bartelt, Christian Stuckenschmidt, Heiner |
| author_facet | Tschalzev, Andrej Purucker, Lennart Lüdtke, Stefan Hutter, Frank Bartelt, Christian Stuckenschmidt, Heiner |
| contents | Data repositories have accumulated a large number of tabular datasets from various domains. Machine Learning researchers are actively using these datasets to evaluate novel approaches. Consequently, data repositories have an important standing in tabular data research. They not only host datasets but also provide information on how to use them in supervised learning tasks. In this paper, we argue that, despite great achievements in usability, the unreflected usage of datasets from data repositories may have led to reduced research quality and scientific rigor. We present examples from prominent recent studies that illustrate the problematic use of datasets from OpenML, a large data repository for tabular data. Our illustrations help users of data repositories avoid falling into the traps of (1) using suboptimal model selection strategies, (2) overlooking strong baselines, and (3) inappropriate preprocessing. In response, we discuss possible solutions for how data repositories can prevent the inappropriate use of datasets and become the cornerstones for improved overall quality of empirical research studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_09159 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Unreflected Use of Tabular Data Repositories Can Undermine Research Quality Tschalzev, Andrej Purucker, Lennart Lüdtke, Stefan Hutter, Frank Bartelt, Christian Stuckenschmidt, Heiner Machine Learning Data repositories have accumulated a large number of tabular datasets from various domains. Machine Learning researchers are actively using these datasets to evaluate novel approaches. Consequently, data repositories have an important standing in tabular data research. They not only host datasets but also provide information on how to use them in supervised learning tasks. In this paper, we argue that, despite great achievements in usability, the unreflected usage of datasets from data repositories may have led to reduced research quality and scientific rigor. We present examples from prominent recent studies that illustrate the problematic use of datasets from OpenML, a large data repository for tabular data. Our illustrations help users of data repositories avoid falling into the traps of (1) using suboptimal model selection strategies, (2) overlooking strong baselines, and (3) inappropriate preprocessing. In response, we discuss possible solutions for how data repositories can prevent the inappropriate use of datasets and become the cornerstones for improved overall quality of empirical research studies. |
| title | Unreflected Use of Tabular Data Repositories Can Undermine Research Quality |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2503.09159 |