StreamFlow: cross-breeding cloud with HPC
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2020
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916313658556416 |
|---|---|
| author | Colonnelli, Iacopo Cantalupo, Barbara Merelli, Ivan Aldinucci, Marco |
| author_facet | Colonnelli, Iacopo Cantalupo, Barbara Merelli, Ivan Aldinucci, Marco |
| contents | Workflows are among the most commonly used tools in a variety of execution environments. Many of them target a specific environment; few of them make it possible to execute an entire workflow in different environments, e.g. Kubernetes and batch clusters. We present a novel approach to workflow execution, called StreamFlow, that complements the workflow graph with the declarative description of potentially complex execution environments, and that makes it possible the execution onto multiple sites not sharing a common data space. StreamFlow is then exemplified on a novel bioinformatics pipeline for single-cell transcriptomic data analysis workflow. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2002_01558 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | StreamFlow: cross-breeding cloud with HPC Colonnelli, Iacopo Cantalupo, Barbara Merelli, Ivan Aldinucci, Marco Distributed, Parallel, and Cluster Computing D.1.3; D.3.2; C.1.3 Workflows are among the most commonly used tools in a variety of execution environments. Many of them target a specific environment; few of them make it possible to execute an entire workflow in different environments, e.g. Kubernetes and batch clusters. We present a novel approach to workflow execution, called StreamFlow, that complements the workflow graph with the declarative description of potentially complex execution environments, and that makes it possible the execution onto multiple sites not sharing a common data space. StreamFlow is then exemplified on a novel bioinformatics pipeline for single-cell transcriptomic data analysis workflow. |
| title | StreamFlow: cross-breeding cloud with HPC |
| topic | Distributed, Parallel, and Cluster Computing D.1.3; D.3.2; C.1.3 |
| url | https://arxiv.org/abs/2002.01558 |