SuperSONIC: Cloud-Native Infrastructure for ML Inferencing
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912449949597696 |
|---|---|
| author | Kondratyev, Dmitry Riedel, Benedikt Chou, Yuan-Tang Cochran-Branson, Miles Paladino, Noah Schultz, David Liu, Mia Duarte, Javier Harris, Philip Hsu, Shih-Chieh |
| author_facet | Kondratyev, Dmitry Riedel, Benedikt Chou, Yuan-Tang Cochran-Branson, Miles Paladino, Noah Schultz, David Liu, Mia Duarte, Javier Harris, Philip Hsu, Shih-Chieh |
| contents | The increasing computational demand from growing data rates and complex machine learning (ML) algorithms in large-scale scientific experiments has driven the adoption of the Services for Optimized Network Inference on Coprocessors (SONIC) approach. SONIC accelerates ML inference by offloading it to local or remote coprocessors to optimize resource utilization. Leveraging its portability to different types of coprocessors, SONIC enhances data processing and model deployment efficiency for cutting-edge research in high energy physics (HEP) and multi-messenger astrophysics (MMA). We developed the SuperSONIC project, a scalable server infrastructure for SONIC, enabling the deployment of computationally intensive tasks to Kubernetes clusters equipped with graphics processing units (GPUs). Using NVIDIA Triton Inference Server, SuperSONIC decouples client workflows from server infrastructure, standardizing communication, optimizing throughput, load balancing, and monitoring. SuperSONIC has been successfully deployed for the CMS and ATLAS experiments at the CERN Large Hadron Collider (LHC), the IceCube Neutrino Observatory (IceCube), and the Laser Interferometer Gravitational-Wave Observatory (LIGO) and tested on Kubernetes clusters at Purdue University, the National Research Platform (NRP), and the University of Chicago. SuperSONIC addresses the challenges of the Cloud-native era by providing a reusable, configurable framework that enhances the efficiency of accelerator-based inference deployment across diverse scientific domains and industries. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_20657 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SuperSONIC: Cloud-Native Infrastructure for ML Inferencing Kondratyev, Dmitry Riedel, Benedikt Chou, Yuan-Tang Cochran-Branson, Miles Paladino, Noah Schultz, David Liu, Mia Duarte, Javier Harris, Philip Hsu, Shih-Chieh Distributed, Parallel, and Cluster Computing High Energy Physics - Experiment Instrumentation and Detectors The increasing computational demand from growing data rates and complex machine learning (ML) algorithms in large-scale scientific experiments has driven the adoption of the Services for Optimized Network Inference on Coprocessors (SONIC) approach. SONIC accelerates ML inference by offloading it to local or remote coprocessors to optimize resource utilization. Leveraging its portability to different types of coprocessors, SONIC enhances data processing and model deployment efficiency for cutting-edge research in high energy physics (HEP) and multi-messenger astrophysics (MMA). We developed the SuperSONIC project, a scalable server infrastructure for SONIC, enabling the deployment of computationally intensive tasks to Kubernetes clusters equipped with graphics processing units (GPUs). Using NVIDIA Triton Inference Server, SuperSONIC decouples client workflows from server infrastructure, standardizing communication, optimizing throughput, load balancing, and monitoring. SuperSONIC has been successfully deployed for the CMS and ATLAS experiments at the CERN Large Hadron Collider (LHC), the IceCube Neutrino Observatory (IceCube), and the Laser Interferometer Gravitational-Wave Observatory (LIGO) and tested on Kubernetes clusters at Purdue University, the National Research Platform (NRP), and the University of Chicago. SuperSONIC addresses the challenges of the Cloud-native era by providing a reusable, configurable framework that enhances the efficiency of accelerator-based inference deployment across diverse scientific domains and industries. |
| title | SuperSONIC: Cloud-Native Infrastructure for ML Inferencing |
| topic | Distributed, Parallel, and Cluster Computing High Energy Physics - Experiment Instrumentation and Detectors |
| url | https://arxiv.org/abs/2506.20657 |