An Error-Bounded Lossy Compression Method with Bit-Adaptive Quantization for Particle Data
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914741126955008 |
|---|---|
| author | Ren, Congrong Di, Sheng Zhang, Longtao Zhao, Kai Guo, Hanqi |
| author_facet | Ren, Congrong Di, Sheng Zhang, Longtao Zhao, Kai Guo, Hanqi |
| contents | This paper presents error-bounded lossy compression tailored for particle datasets from diverse scientific applications in cosmology, fluid dynamics, and fusion energy sciences. As today's high-performance computing capabilities advance, these datasets often reach trillions of points, posing significant visualization, analysis, and storage challenges. While error-bounded lossy compression makes it possible to represent floating-point values with strict pointwise accuracy guarantees, the lack of correlations in particle data's storage ordering often limits the compression ratio. Inspired by quantization-encoding schemes in SZ lossy compressors, we dynamically determine the number of bits to encode particles of the dataset to increase the compression ratio. Specifically, we utilize a k-d tree to partition particles into subregions and generate ``bit boxes'' centered at particles for each subregion to encode their positions. These bit boxes ensure error control while reducing the bit count used for compression. We comprehensively evaluate our method against state-of-the-art compressors on cosmology, fluid dynamics, and fusion plasma datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_02826 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An Error-Bounded Lossy Compression Method with Bit-Adaptive Quantization for Particle Data Ren, Congrong Di, Sheng Zhang, Longtao Zhao, Kai Guo, Hanqi Information Theory Instrumentation and Methods for Astrophysics Graphics This paper presents error-bounded lossy compression tailored for particle datasets from diverse scientific applications in cosmology, fluid dynamics, and fusion energy sciences. As today's high-performance computing capabilities advance, these datasets often reach trillions of points, posing significant visualization, analysis, and storage challenges. While error-bounded lossy compression makes it possible to represent floating-point values with strict pointwise accuracy guarantees, the lack of correlations in particle data's storage ordering often limits the compression ratio. Inspired by quantization-encoding schemes in SZ lossy compressors, we dynamically determine the number of bits to encode particles of the dataset to increase the compression ratio. Specifically, we utilize a k-d tree to partition particles into subregions and generate ``bit boxes'' centered at particles for each subregion to encode their positions. These bit boxes ensure error control while reducing the bit count used for compression. We comprehensively evaluate our method against state-of-the-art compressors on cosmology, fluid dynamics, and fusion plasma datasets. |
| title | An Error-Bounded Lossy Compression Method with Bit-Adaptive Quantization for Particle Data |
| topic | Information Theory Instrumentation and Methods for Astrophysics Graphics |
| url | https://arxiv.org/abs/2404.02826 |