Ichnos: A Carbon Footprint Estimator for Scientific Workflows
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915431193772032 |
|---|---|
| author | West, Kathleen Reid, Magnus Elkhatib, Yehia Thamsen, Lauritz |
| author_facet | West, Kathleen Reid, Magnus Elkhatib, Yehia Thamsen, Lauritz |
| contents | Scientific workflows facilitate the automation of data analysis, and are used to process increasing amounts of data. Therefore, they tend to be resource-intensive and long-running, leading to significant energy consumption and carbon emissions. With ever-increasing emissions from the ICT sector, it is crucial to quantify and understand the carbon footprint of scientific workflows. However, existing tooling requires significant effort from users - such as setting up power monitoring before executing workloads, or translating monitored metrics into the carbon footprints post-execution. In this paper, we introduce a system to estimate the carbon footprint of Nextflow scientific workflows that enables post-hoc estimation based on existing workflow traces, power models for computational resources utilised, and carbon intensity data aligned with the execution time. We discuss our automated power modelling approach, and compare it with commonly used estimation methodologies. Furthermore, we exemplify several potential use cases and evaluate our energy consumption estimation approach, finding its estimation error to be between 3.9-10.3%, outperforming both baseline methodologies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_12456 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Ichnos: A Carbon Footprint Estimator for Scientific Workflows West, Kathleen Reid, Magnus Elkhatib, Yehia Thamsen, Lauritz Distributed, Parallel, and Cluster Computing Scientific workflows facilitate the automation of data analysis, and are used to process increasing amounts of data. Therefore, they tend to be resource-intensive and long-running, leading to significant energy consumption and carbon emissions. With ever-increasing emissions from the ICT sector, it is crucial to quantify and understand the carbon footprint of scientific workflows. However, existing tooling requires significant effort from users - such as setting up power monitoring before executing workloads, or translating monitored metrics into the carbon footprints post-execution. In this paper, we introduce a system to estimate the carbon footprint of Nextflow scientific workflows that enables post-hoc estimation based on existing workflow traces, power models for computational resources utilised, and carbon intensity data aligned with the execution time. We discuss our automated power modelling approach, and compare it with commonly used estimation methodologies. Furthermore, we exemplify several potential use cases and evaluate our energy consumption estimation approach, finding its estimation error to be between 3.9-10.3%, outperforming both baseline methodologies. |
| title | Ichnos: A Carbon Footprint Estimator for Scientific Workflows |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2411.12456 |