Saved in:
| Main Author: | |
|---|---|
| Format: | Recurso digital |
| Language: | |
| Published: |
Zenodo
2025
|
| Online Access: | https://doi.org/10.5281/zenodo.17768039 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Table of Contents:
- <h2>What's new in v1.1</h2> <p>v1.1 – Parameterised, and Safer Merging This release turns <code>csv-merge-windows</code> from a "drop in folder and double-click" tool into a small but flexible CSV utility, still optimised for large GBD-style datasets.</p> <p>Key themes:</p> <ul> <li><strong>Parameterised</strong>: choose which files to merge, where to write output, and how large Excel-friendly chunks should be.</li> <li><strong>Excel-aware</strong>: optional automatic splitting of the merged output into ≤1,000,000-row chunks.</li> <li><strong>Safer</strong>: optional header validation and dry-run mode so you can inspect what will happen before writing anything.</li> </ul> <h2>Added</h2> <h2>1. Parameterised PowerShell script</h2> <p><code>merge_csvs.ps1</code> now accepts optional parameters:</p> <ul> <li><p><code>-Pattern</code> – file pattern to merge (default: <code>*.csv</code>)</p> <pre><code>.\merge_csvs.ps1 -Pattern 'IHME-GBD_2023_DATA-*.csv' </code></pre> </li> </ul> <p>-Output – base name of the merged file (default: merged.csv) .\merge_csvs.ps1 -Output 'gbd_merged.csv'</p> <p>-AddSourceColumn – when supplied, adds a final column containing the source filename for each row: .\merge_csvs.ps1 -AddSourceColumn</p> <p>-DryRun – prints a plan (which files, detected header, approximate line counts) but does not create any output files: .\merge_csvs.ps1 -DryRun</p> <h2>2. Excel-friendly chunking</h2> <ul> <li>Large merged files can exceed Excel's per-sheet row limit (~1,048,576 rows). v1.1 introduces an optional chunking mode: New parameter -MaxRowsPerFile (default: no limit). If set, the script will create multiple outputs: .\merge_csvs.ps1 -Output 'gbd_merged.csv' -MaxRowsPerFile 1000000</li> </ul> <h2>3. Header consistency check</h2> <p>New parameters: -ValidateHeaders – log a warning for files whose header doesn't match. -StrictHeaders – throw and stop if any file's header differs.</p> <h3>Warn but continue</h3> <p>.\merge_csvs.ps1 -ValidateHeaders</p> <h3>Fail fast on header mismatch</h3> <p>.\merge_csvs.ps1 -StrictHeaders</p>