Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.01754 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909412352851968 |
|---|---|
| author | Luo, Xihaier Lurvey, Samuel Huang, Yi Ren, Yihui Huang, Jin Yoon, Byung-Jun |
| author_facet | Luo, Xihaier Lurvey, Samuel Huang, Yi Ren, Yihui Huang, Jin Yoon, Byung-Jun |
| contents | High-energy, large-scale particle colliders in nuclear and high-energy physics generate data at extraordinary rates, reaching up to $1$ terabyte and several petabytes per second, respectively. The development of real-time, high-throughput data compression algorithms capable of reducing this data to manageable sizes for permanent storage is of paramount importance. A unique characteristic of the tracking detector data is the extreme sparsity of particle trajectories in space, with an occupancy rate ranging from approximately $10^{-6}$ to $10\%$. Furthermore, for downstream tasks, a continuous representation of this data is often more useful than a voxel-based, discrete representation due to the inherently continuous nature of the signals involved. To address these challenges, we propose a novel approach using implicit neural representations for data learning and compression. We also introduce an importance sampling technique to accelerate the network training process. Our method is competitive with traditional compression algorithms, such as MGARD, SZ, and ZFP, while offering significant speed-ups and maintaining negligible accuracy loss through our importance sampling strategy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_01754 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Efficient Compression of Sparse Accelerator Data Using Implicit Neural Representations and Importance Sampling Luo, Xihaier Lurvey, Samuel Huang, Yi Ren, Yihui Huang, Jin Yoon, Byung-Jun Artificial Intelligence High-energy, large-scale particle colliders in nuclear and high-energy physics generate data at extraordinary rates, reaching up to $1$ terabyte and several petabytes per second, respectively. The development of real-time, high-throughput data compression algorithms capable of reducing this data to manageable sizes for permanent storage is of paramount importance. A unique characteristic of the tracking detector data is the extreme sparsity of particle trajectories in space, with an occupancy rate ranging from approximately $10^{-6}$ to $10\%$. Furthermore, for downstream tasks, a continuous representation of this data is often more useful than a voxel-based, discrete representation due to the inherently continuous nature of the signals involved. To address these challenges, we propose a novel approach using implicit neural representations for data learning and compression. We also introduce an importance sampling technique to accelerate the network training process. Our method is competitive with traditional compression algorithms, such as MGARD, SZ, and ZFP, while offering significant speed-ups and maintaining negligible accuracy loss through our importance sampling strategy. |
| title | Efficient Compression of Sparse Accelerator Data Using Implicit Neural Representations and Importance Sampling |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2412.01754 |