Compendium Manager: a tool for coordination of workflow management instances for bulk data processing in Python
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910947977723904 |
|---|---|
| author | Abdill, Richard J. Blekhman, Ran |
| author_facet | Abdill, Richard J. Blekhman, Ran |
| contents | Compendium Manager is a command-line tool written in Python to automate the provisioning, launch, and evaluation of bioinformatics pipelines. Although workflow management tools such as Snakemake and Nextflow enable users to automate the processing of samples within a single sequencing project, integrating many datasets in bulk requires launching and monitoring hundreds or thousands of pipelines. We present the Compendium Manager, a lightweight command-line tool to enable launching and monitoring analysis pipelines at scale. The tool can gauge progress through a list of projects, load results into a shared database, and record detailed processing metrics for later evaluation and reproducibility. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_11385 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Compendium Manager: a tool for coordination of workflow management instances for bulk data processing in Python Abdill, Richard J. Blekhman, Ran Genomics Software Engineering 92-04 J.3 Compendium Manager is a command-line tool written in Python to automate the provisioning, launch, and evaluation of bioinformatics pipelines. Although workflow management tools such as Snakemake and Nextflow enable users to automate the processing of samples within a single sequencing project, integrating many datasets in bulk requires launching and monitoring hundreds or thousands of pipelines. We present the Compendium Manager, a lightweight command-line tool to enable launching and monitoring analysis pipelines at scale. The tool can gauge progress through a list of projects, load results into a shared database, and record detailed processing metrics for later evaluation and reproducibility. |
| title | Compendium Manager: a tool for coordination of workflow management instances for bulk data processing in Python |
| topic | Genomics Software Engineering 92-04 J.3 |
| url | https://arxiv.org/abs/2505.11385 |