The Real-Time Data Processor Framework for Data Handling and Analysis of High-Energy Instruments
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912689829183488 |
|---|---|
| author | Bulgarelli, A. Parmiggiani, N. Castaldini, L. Falco, R. Di Piano, A. Fioretti, V. Panebianco, G. Rizzo, A. |
| author_facet | Bulgarelli, A. Parmiggiani, N. Castaldini, L. Falco, R. Di Piano, A. Fioretti, V. Panebianco, G. Rizzo, A. |
| contents | We implemented a real-time data processor (rta-dp) framework that can be used to develop real-time analysis pipelines and data handling systems to manage high-throughput data streams with distributed applications in the context of ground and space astrophysical projects and high-energy instruments. The rta-dp is based on the ZeroMQ in-memory communication framework to receive input data, share data between distributed processes, and send or receive commands and pipeline configuration. The rta-dp framework has a flexible architecture that allows the implementation of distributed analysis systems customized to the requirements of several scenarios. The rta-dp framework also provides monitoring capabilities for the running processes and sends housekeeping, logging, alarms, and informative messages that a monitoring process can acquire. We are using the rta-dp in several contexts, such as acquiring and processing data from X-ray detectors to the data quality system of the ASTRI Project, as well as reprocessing and archiving data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_03760 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Real-Time Data Processor Framework for Data Handling and Analysis of High-Energy Instruments Bulgarelli, A. Parmiggiani, N. Castaldini, L. Falco, R. Di Piano, A. Fioretti, V. Panebianco, G. Rizzo, A. Instrumentation and Methods for Astrophysics We implemented a real-time data processor (rta-dp) framework that can be used to develop real-time analysis pipelines and data handling systems to manage high-throughput data streams with distributed applications in the context of ground and space astrophysical projects and high-energy instruments. The rta-dp is based on the ZeroMQ in-memory communication framework to receive input data, share data between distributed processes, and send or receive commands and pipeline configuration. The rta-dp framework has a flexible architecture that allows the implementation of distributed analysis systems customized to the requirements of several scenarios. The rta-dp framework also provides monitoring capabilities for the running processes and sends housekeeping, logging, alarms, and informative messages that a monitoring process can acquire. We are using the rta-dp in several contexts, such as acquiring and processing data from X-ray detectors to the data quality system of the ASTRI Project, as well as reprocessing and archiving data. |
| title | The Real-Time Data Processor Framework for Data Handling and Analysis of High-Energy Instruments |
| topic | Instrumentation and Methods for Astrophysics |
| url | https://arxiv.org/abs/2511.03760 |