Exploring the Potential of Carbon-Aware Execution for Scientific Workflows
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909542184386560 |
|---|---|
| author | West, Kathleen Lehmann, Fabian Bountris, Vasilis Leser, Ulf Elkhatib, Yehia Thamsen, Lauritz |
| author_facet | West, Kathleen Lehmann, Fabian Bountris, Vasilis Leser, Ulf Elkhatib, Yehia Thamsen, Lauritz |
| contents | Scientific workflows are widely used to automate scientific data analysis and often involve processing large quantities of data on compute clusters. As such, their execution tends to be long-running and resource intensive, leading to significant energy consumption and carbon emissions.
Meanwhile, a wealth of carbon-aware computing methods have been proposed, yet little work has focused specifically on scientific workflows, even though they present a substantial opportunity for carbon-aware computing because they are inherently delay tolerant, efficiently interruptible, and highly scalable.
In this study, we demonstrate the potential for carbon-aware workflow execution. For this, we estimate the carbon footprint of two real-world Nextflow workflows executed on cluster infrastructure. We use a linear power model for energy consumption estimates and real-world average and marginal CI data for two regions. We evaluate the impact of carbon-aware temporal shifting, pausing and resuming, and resource scaling. Our findings highlight significant potential for reducing emissions of workflows and workflow tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_13705 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploring the Potential of Carbon-Aware Execution for Scientific Workflows West, Kathleen Lehmann, Fabian Bountris, Vasilis Leser, Ulf Elkhatib, Yehia Thamsen, Lauritz Distributed, Parallel, and Cluster Computing Scientific workflows are widely used to automate scientific data analysis and often involve processing large quantities of data on compute clusters. As such, their execution tends to be long-running and resource intensive, leading to significant energy consumption and carbon emissions. Meanwhile, a wealth of carbon-aware computing methods have been proposed, yet little work has focused specifically on scientific workflows, even though they present a substantial opportunity for carbon-aware computing because they are inherently delay tolerant, efficiently interruptible, and highly scalable. In this study, we demonstrate the potential for carbon-aware workflow execution. For this, we estimate the carbon footprint of two real-world Nextflow workflows executed on cluster infrastructure. We use a linear power model for energy consumption estimates and real-world average and marginal CI data for two regions. We evaluate the impact of carbon-aware temporal shifting, pausing and resuming, and resource scaling. Our findings highlight significant potential for reducing emissions of workflows and workflow tasks. |
| title | Exploring the Potential of Carbon-Aware Execution for Scientific Workflows |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2503.13705 |