Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.16177 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913515749507072 |
|---|---|
| author | Scomparin, Luca Caselle, Michele Garcia, Andrea Santamaria Xu, Chenran Blomley, Edmund Dritschler, Timo Mochihashi, Akira Schuh, Marcel Steinmann, Johannes L. Bründermann, Erik Kopmann, Andreas Becker, Jürgen Müller, Anke-Susanne Weber, Marc |
| author_facet | Scomparin, Luca Caselle, Michele Garcia, Andrea Santamaria Xu, Chenran Blomley, Edmund Dritschler, Timo Mochihashi, Akira Schuh, Marcel Steinmann, Johannes L. Bründermann, Erik Kopmann, Andreas Becker, Jürgen Müller, Anke-Susanne Weber, Marc |
| contents | The commissioning and operation of future large-scale scientific experiments will challenge current tuning and control methods. Reinforcement learning (RL) algorithms are a promising solution thanks to their capability of autonomously tackling a control problem based on a task parameterized by a reward function. The conventionally utilized machine learning (ML) libraries are not intended for microsecond latency applications, as they mostly optimize for throughput performance. On the other hand, most of the programmable logic implementations are meant for computation acceleration, not being intended to work in a real-time environment. To overcome these limitations of current implementations, RL needs to be deployed on-the-edge, i.e. on to the device gathering the training data. In this paper we present the design and deployment of an experience accumulator system in a particle accelerator. In this system deep-RL algorithms run using hardware acceleration and act within a few microseconds, enabling the use of RL for control of ultra-fast phenomena. The training is performed offline to reduce the number of operations carried out on the acceleration hardware. The proposed architecture was tested in real experimental conditions at the Karlsruhe research accelerator (KARA), serving also as a synchrotron light source, where the system was used to control induced horizontal betatron oscillations in real-time. The results showed a performance comparable to the commercial feedback system available at the accelerator, proving the viability and potential of this approach. Due to the self-learning and reconfiguration capability of this implementation, its seamless application to other control problems is possible. Applications range from particle accelerators to large-scale research and industrial facilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_16177 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Microsecond-Latency Feedback at a Particle Accelerator by Online Reinforcement Learning on Hardware Scomparin, Luca Caselle, Michele Garcia, Andrea Santamaria Xu, Chenran Blomley, Edmund Dritschler, Timo Mochihashi, Akira Schuh, Marcel Steinmann, Johannes L. Bründermann, Erik Kopmann, Andreas Becker, Jürgen Müller, Anke-Susanne Weber, Marc Accelerator Physics High Energy Physics - Experiment Instrumentation and Detectors The commissioning and operation of future large-scale scientific experiments will challenge current tuning and control methods. Reinforcement learning (RL) algorithms are a promising solution thanks to their capability of autonomously tackling a control problem based on a task parameterized by a reward function. The conventionally utilized machine learning (ML) libraries are not intended for microsecond latency applications, as they mostly optimize for throughput performance. On the other hand, most of the programmable logic implementations are meant for computation acceleration, not being intended to work in a real-time environment. To overcome these limitations of current implementations, RL needs to be deployed on-the-edge, i.e. on to the device gathering the training data. In this paper we present the design and deployment of an experience accumulator system in a particle accelerator. In this system deep-RL algorithms run using hardware acceleration and act within a few microseconds, enabling the use of RL for control of ultra-fast phenomena. The training is performed offline to reduce the number of operations carried out on the acceleration hardware. The proposed architecture was tested in real experimental conditions at the Karlsruhe research accelerator (KARA), serving also as a synchrotron light source, where the system was used to control induced horizontal betatron oscillations in real-time. The results showed a performance comparable to the commercial feedback system available at the accelerator, proving the viability and potential of this approach. Due to the self-learning and reconfiguration capability of this implementation, its seamless application to other control problems is possible. Applications range from particle accelerators to large-scale research and industrial facilities. |
| title | Microsecond-Latency Feedback at a Particle Accelerator by Online Reinforcement Learning on Hardware |
| topic | Accelerator Physics High Energy Physics - Experiment Instrumentation and Detectors |
| url | https://arxiv.org/abs/2409.16177 |