github.com/DataBiosphere/xvcfmerge/xvcfmerge
Fuente:
Zenodo
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Recurso digital |
| Pubblicazione: |
Zenodo
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866901690719928320 |
|---|---|
| author | Brian Hannafious Lon Blauvelt |
| author_facet | Brian Hannafious Lon Blauvelt |
| contents | <h1>xvcfmerge</h1> <p>Define a WDL workflow to merge jointly called VCF files. This workflow accepts input as array of reference URIs. The reference URIs should be either Google Storage URLs or <a href="https://support.terra.bio/hc/en-us/articles/360039330211-Data-Access-with-the-GA4GH-Data-Repository-Service-DRS-">DRS</a> URIs.</p> <h2>Description</h2> <p><a href="https://github.com/DataBiosphere/xsamtools">xsamtools</a> is used to perform the merge. Each input VCF should describe a single chromosome and should have equivalent headers.</p> <p>In order to keep hard disk requirements small and improve performance, input/output objects are streamed directly from cloud storage using FIFO queues (named pipes).</p> <p>An appropriate workflow configuration is ~16GB RAM, ~8 CPUs, and standard storage.</p> <h2>Runtime</h2> <p>Runtime varies with the number of VCFs to merge, the number of samples, runtime configuration, and other factors. The table below may be helpful to understand the expected runtime for your workflow.</p> <p>| cpu | memory | number of vcfs | average size | samples | runtime | | --- | ------ | -------------- | ------------ | ------- | --------- | | 2 | 2GB | 2 | 2GB | 2 | 0.9 hours | | 8 | 64GB | 2 | 2GB | 2 | 0.7 hours | | 8 | 64GB | 2 | 5GB | 5370 | 4 hours | | 8 | 64GB | 5 | 5GB | 4100 | 18 hours |</p> <h2>Tests</h2> <p>Test workflows are executed locally with <a href="https://github.com/chanzuckerberg/miniwdl">miniwdl</a>, and require access to the <a href="https://terra.biodatacatalyst.nhlbi.nih.gov/#workspaces/firecloud-cgl/terra-notebook-utils-tests">terra-notebook-utils-tests</a> Terra workspace and GCP authorization.</p> <p>To run basic tests,</p> <pre><code>docker login gcloud auth login make test </code></pre> <p>Note that the WDL built-in function size() does not function in miniwdl, which is used in the makefile, when working with gs URIs which may lead to a failing test unless lines involving that function are commented out. size() functions correctly on Cromwell and is a necessary inclusion for GCP-based backends including Terra.</p> <p>More robust tests against DRS resolved objects should be executed prior to release with</p> <pre><code>gcloud auth application-default login make test_drs </code></pre> <p>Please contact Lon (lblauvel at ucsc dot edu) for data and workspace access issues.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_15610743 |
| institution | Zenodo |
| language | |
| publishDate | 2022 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | github.com/DataBiosphere/xvcfmerge/xvcfmerge Brian Hannafious Lon Blauvelt utilities xsamtools xvcf <h1>xvcfmerge</h1> <p>Define a WDL workflow to merge jointly called VCF files. This workflow accepts input as array of reference URIs. The reference URIs should be either Google Storage URLs or <a href="https://support.terra.bio/hc/en-us/articles/360039330211-Data-Access-with-the-GA4GH-Data-Repository-Service-DRS-">DRS</a> URIs.</p> <h2>Description</h2> <p><a href="https://github.com/DataBiosphere/xsamtools">xsamtools</a> is used to perform the merge. Each input VCF should describe a single chromosome and should have equivalent headers.</p> <p>In order to keep hard disk requirements small and improve performance, input/output objects are streamed directly from cloud storage using FIFO queues (named pipes).</p> <p>An appropriate workflow configuration is ~16GB RAM, ~8 CPUs, and standard storage.</p> <h2>Runtime</h2> <p>Runtime varies with the number of VCFs to merge, the number of samples, runtime configuration, and other factors. The table below may be helpful to understand the expected runtime for your workflow.</p> <p>| cpu | memory | number of vcfs | average size | samples | runtime | | --- | ------ | -------------- | ------------ | ------- | --------- | | 2 | 2GB | 2 | 2GB | 2 | 0.9 hours | | 8 | 64GB | 2 | 2GB | 2 | 0.7 hours | | 8 | 64GB | 2 | 5GB | 5370 | 4 hours | | 8 | 64GB | 5 | 5GB | 4100 | 18 hours |</p> <h2>Tests</h2> <p>Test workflows are executed locally with <a href="https://github.com/chanzuckerberg/miniwdl">miniwdl</a>, and require access to the <a href="https://terra.biodatacatalyst.nhlbi.nih.gov/#workspaces/firecloud-cgl/terra-notebook-utils-tests">terra-notebook-utils-tests</a> Terra workspace and GCP authorization.</p> <p>To run basic tests,</p> <pre><code>docker login gcloud auth login make test </code></pre> <p>Note that the WDL built-in function size() does not function in miniwdl, which is used in the makefile, when working with gs URIs which may lead to a failing test unless lines involving that function are commented out. size() functions correctly on Cromwell and is a necessary inclusion for GCP-based backends including Terra.</p> <p>More robust tests against DRS resolved objects should be executed prior to release with</p> <pre><code>gcloud auth application-default login make test_drs </code></pre> <p>Please contact Lon (lblauvel at ucsc dot edu) for data and workspace access issues.</p> |
| title | github.com/DataBiosphere/xvcfmerge/xvcfmerge |
| topic | utilities xsamtools xvcf |
| url | https://doi.org/10.5281/zenodo.15610743 |