A Survey on Error-Bounded Lossy Compression for Scientific Datasets
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910907135688704 |
|---|---|
| author | Di, Sheng Liu, Jinyang Zhao, Kai Liang, Xin Underwood, Robert Zhang, Zhaorui Shah, Milan Huang, Yafan Huang, Jiajun Yu, Xiaodong Ren, Congrong Guo, Hanqi Wilkins, Grant Tao, Dingwen Tian, Jiannan Jin, Sian Jian, Zizhe Wang, Daoce Rahman, MD Hasanur Zhang, Boyuan Song, Shihui Calhoun, Jon C. Li, Guanpeng Yoshii, Kazutomo Alharthi, Khalid Ayed Cappello, Franck |
| author_facet | Di, Sheng Liu, Jinyang Zhao, Kai Liang, Xin Underwood, Robert Zhang, Zhaorui Shah, Milan Huang, Yafan Huang, Jiajun Yu, Xiaodong Ren, Congrong Guo, Hanqi Wilkins, Grant Tao, Dingwen Tian, Jiannan Jin, Sian Jian, Zizhe Wang, Daoce Rahman, MD Hasanur Zhang, Boyuan Song, Shihui Calhoun, Jon C. Li, Guanpeng Yoshii, Kazutomo Alharthi, Khalid Ayed Cappello, Franck |
| contents | Error-bounded lossy compression has been effective in significantly reducing the data storage/transfer burden while preserving the reconstructed data fidelity very well. Many error-bounded lossy compressors have been developed for a wide range of parallel and distributed use cases for years. They are designed with distinct compression models and principles, such that each of them features particular pros and cons. In this paper we provide a comprehensive survey of emerging error-bounded lossy compression techniques. The key contribution is fourfold. (1) We summarize a novel taxonomy of lossy compression into 6 classic models. (2) We provide a comprehensive survey of 10 commonly used compression components/modules. (3) We summarized pros and cons of 46 state-of-the-art lossy compressors and present how state-of-the-art compressors are designed based on different compression techniques. (4) We discuss how customized compressors are designed for specific scientific applications and use-cases. We believe this survey is useful to multiple communities including scientific applications, high-performance computing, lossy compression, and big data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_02840 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Survey on Error-Bounded Lossy Compression for Scientific Datasets Di, Sheng Liu, Jinyang Zhao, Kai Liang, Xin Underwood, Robert Zhang, Zhaorui Shah, Milan Huang, Yafan Huang, Jiajun Yu, Xiaodong Ren, Congrong Guo, Hanqi Wilkins, Grant Tao, Dingwen Tian, Jiannan Jin, Sian Jian, Zizhe Wang, Daoce Rahman, MD Hasanur Zhang, Boyuan Song, Shihui Calhoun, Jon C. Li, Guanpeng Yoshii, Kazutomo Alharthi, Khalid Ayed Cappello, Franck Distributed, Parallel, and Cluster Computing Error-bounded lossy compression has been effective in significantly reducing the data storage/transfer burden while preserving the reconstructed data fidelity very well. Many error-bounded lossy compressors have been developed for a wide range of parallel and distributed use cases for years. They are designed with distinct compression models and principles, such that each of them features particular pros and cons. In this paper we provide a comprehensive survey of emerging error-bounded lossy compression techniques. The key contribution is fourfold. (1) We summarize a novel taxonomy of lossy compression into 6 classic models. (2) We provide a comprehensive survey of 10 commonly used compression components/modules. (3) We summarized pros and cons of 46 state-of-the-art lossy compressors and present how state-of-the-art compressors are designed based on different compression techniques. (4) We discuss how customized compressors are designed for specific scientific applications and use-cases. We believe this survey is useful to multiple communities including scientific applications, high-performance computing, lossy compression, and big data. |
| title | A Survey on Error-Bounded Lossy Compression for Scientific Datasets |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2404.02840 |