QuaRs: A Transform for Better Lossless Compression of Integers
Fuente:
arXiv
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909463880925184 |
|---|---|
| author | Matt, Jonas G. |
| author_facet | Matt, Jonas G. |
| contents | The rise of integer-valued data, partly driven by the Internet of Things (IoT), has increased demand for efficient compression methods to reduce storage and transmission costs. Existing, speed-oriented methods rely on the ``smaller-numbers-less-bits'' principle, assuming unimodal distributions centered around zero. This assumption is often violated in practice, leading to suboptimal compression. We propose QuaRs, a transformation that reshapes arbitrary distributions into unimodal ones centered around zero, improving compatibility with fast integer compression methods. QuaRs remaps data based on quantiles, assigning smaller magnitudes to frequent values. The method is fast, invertible, and has sub-quadratic complexity. QuaRs enhances compression efficiency, even for challenging distributions, while integrating seamlessly with existing techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_12929 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | QuaRs: A Transform for Better Lossless Compression of Integers Matt, Jonas G. Data Structures and Algorithms The rise of integer-valued data, partly driven by the Internet of Things (IoT), has increased demand for efficient compression methods to reduce storage and transmission costs. Existing, speed-oriented methods rely on the ``smaller-numbers-less-bits'' principle, assuming unimodal distributions centered around zero. This assumption is often violated in practice, leading to suboptimal compression. We propose QuaRs, a transformation that reshapes arbitrary distributions into unimodal ones centered around zero, improving compatibility with fast integer compression methods. QuaRs remaps data based on quantiles, assigning smaller magnitudes to frequent values. The method is fast, invertible, and has sub-quadratic complexity. QuaRs enhances compression efficiency, even for challenging distributions, while integrating seamlessly with existing techniques. |
| title | QuaRs: A Transform for Better Lossless Compression of Integers |
| topic | Data Structures and Algorithms |
| url | https://arxiv.org/abs/2501.12929 |