A framework for compressing unstructured scientific data via serialization
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916573802921984 |
|---|---|
| author | Reshniak, Viktor Gong, Qian Archibald, Rick Klasky, Scott Podhorszki, Norbert |
| author_facet | Reshniak, Viktor Gong, Qian Archibald, Rick Klasky, Scott Podhorszki, Norbert |
| contents | We present a general framework for compressing unstructured scientific data with known local connectivity. A common application is simulation data defined on arbitrary finite element meshes. The framework employs a greedy topology preserving reordering of original nodes which allows for seamless integration into existing data processing pipelines. This reordering process depends solely on mesh connectivity and can be performed offline for optimal efficiency. However, the algorithm's greedy nature also supports on-the-fly implementation. The proposed method is compatible with any compression algorithm that leverages spatial correlations within the data. The effectiveness of this approach is demonstrated on a large-scale real dataset using several compression methods, including MGARD, SZ, and ZFP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_08059 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A framework for compressing unstructured scientific data via serialization Reshniak, Viktor Gong, Qian Archibald, Rick Klasky, Scott Podhorszki, Norbert Computer Vision and Pattern Recognition We present a general framework for compressing unstructured scientific data with known local connectivity. A common application is simulation data defined on arbitrary finite element meshes. The framework employs a greedy topology preserving reordering of original nodes which allows for seamless integration into existing data processing pipelines. This reordering process depends solely on mesh connectivity and can be performed offline for optimal efficiency. However, the algorithm's greedy nature also supports on-the-fly implementation. The proposed method is compatible with any compression algorithm that leverages spatial correlations within the data. The effectiveness of this approach is demonstrated on a large-scale real dataset using several compression methods, including MGARD, SZ, and ZFP. |
| title | A framework for compressing unstructured scientific data via serialization |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.08059 |