Pipeline Provenance for Analysis, Evaluation, Trust or Reproducibility
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913324252266496 |
|---|---|
| author | Johnson, Michael A. C. Klöckner, Hans-Rainer Muzafarova, Albina Lackeos, Kristen Champion, David J. Dembska, Marta Schindler, Sirko Paradies, Marcus |
| author_facet | Johnson, Michael A. C. Klöckner, Hans-Rainer Muzafarova, Albina Lackeos, Kristen Champion, David J. Dembska, Marta Schindler, Sirko Paradies, Marcus |
| contents | Data volumes and rates of research infrastructures will continue to increase in the upcoming years and impact how we interact with their final data products. Little of the processed data can be directly investigated and most of it will be automatically processed with as little user interaction as possible. Capturing all necessary information of such processing ensures reproducibility of the final results and generates trust in the entire process. We present PRAETOR, a software suite that enables automated generation, modelling, and analysis of provenance information of Python pipelines. Furthermore, the evaluation of the pipeline performance, based upon a user defined quality matrix in the provenance, enables the first step of machine learning processes, where such information can be fed into dedicated optimisation procedures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_14378 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pipeline Provenance for Analysis, Evaluation, Trust or Reproducibility Johnson, Michael A. C. Klöckner, Hans-Rainer Muzafarova, Albina Lackeos, Kristen Champion, David J. Dembska, Marta Schindler, Sirko Paradies, Marcus Instrumentation and Methods for Astrophysics Data volumes and rates of research infrastructures will continue to increase in the upcoming years and impact how we interact with their final data products. Little of the processed data can be directly investigated and most of it will be automatically processed with as little user interaction as possible. Capturing all necessary information of such processing ensures reproducibility of the final results and generates trust in the entire process. We present PRAETOR, a software suite that enables automated generation, modelling, and analysis of provenance information of Python pipelines. Furthermore, the evaluation of the pipeline performance, based upon a user defined quality matrix in the provenance, enables the first step of machine learning processes, where such information can be fed into dedicated optimisation procedures. |
| title | Pipeline Provenance for Analysis, Evaluation, Trust or Reproducibility |
| topic | Instrumentation and Methods for Astrophysics |
| url | https://arxiv.org/abs/2404.14378 |