Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2312.17372 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913181053485056 |
|---|---|
| author | Xu, Chenwei Hu, Jerry Yao-Chieh Narayanan, Aakaash Thieme, Mattson Nagaslaev, Vladimir Austin, Mark Arnold, Jeremy Berlioz, Jose Hanlet, Pierrick Ibrahim, Aisha Nicklaus, Dennis Mitrevski, Jovan John, Jason Michael St. Pradhan, Gauri Saewert, Andrea Seiya, Kiyomi Schupbach, Brian Thurman-Keup, Randy Tran, Nhan Shi, Rui Ogrenci, Seda Shuping, Alexis Maya-Isabelle Hazelwood, Kyle Liu, Han |
| author_facet | Xu, Chenwei Hu, Jerry Yao-Chieh Narayanan, Aakaash Thieme, Mattson Nagaslaev, Vladimir Austin, Mark Arnold, Jeremy Berlioz, Jose Hanlet, Pierrick Ibrahim, Aisha Nicklaus, Dennis Mitrevski, Jovan John, Jason Michael St. Pradhan, Gauri Saewert, Andrea Seiya, Kiyomi Schupbach, Brian Thurman-Keup, Randy Tran, Nhan Shi, Rui Ogrenci, Seda Shuping, Alexis Maya-Isabelle Hazelwood, Kyle Liu, Han |
| contents | We introduce a novel Proximal Policy Optimization (PPO) algorithm aimed at addressing the challenge of maintaining a uniform proton beam intensity delivery in the Muon to Electron Conversion Experiment (Mu2e) at Fermi National Accelerator Laboratory (Fermilab). Our primary objective is to regulate the spill process to ensure a consistent intensity profile, with the ultimate goal of creating an automated controller capable of providing real-time feedback and calibration of the Spill Regulation System (SRS) parameters on a millisecond timescale. We treat the Mu2e accelerator system as a Markov Decision Process suitable for Reinforcement Learning (RL), utilizing PPO to reduce bias and enhance training stability. A key innovation in our approach is the integration of a neuralized Proportional-Integral-Derivative (PID) controller into the policy function, resulting in a significant improvement in the Spill Duty Factor (SDF) by 13.6%, surpassing the performance of the current PID controller baseline by an additional 1.6%. This paper presents the preliminary offline results based on a differentiable simulator of the Mu2e accelerator. It paves the groundwork for real-time implementations and applications, representing a crucial step towards automated proton beam intensity control for the Mu2e experiment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_17372 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e Xu, Chenwei Hu, Jerry Yao-Chieh Narayanan, Aakaash Thieme, Mattson Nagaslaev, Vladimir Austin, Mark Arnold, Jeremy Berlioz, Jose Hanlet, Pierrick Ibrahim, Aisha Nicklaus, Dennis Mitrevski, Jovan John, Jason Michael St. Pradhan, Gauri Saewert, Andrea Seiya, Kiyomi Schupbach, Brian Thurman-Keup, Randy Tran, Nhan Shi, Rui Ogrenci, Seda Shuping, Alexis Maya-Isabelle Hazelwood, Kyle Liu, Han Machine Learning Artificial Intelligence Accelerator Physics We introduce a novel Proximal Policy Optimization (PPO) algorithm aimed at addressing the challenge of maintaining a uniform proton beam intensity delivery in the Muon to Electron Conversion Experiment (Mu2e) at Fermi National Accelerator Laboratory (Fermilab). Our primary objective is to regulate the spill process to ensure a consistent intensity profile, with the ultimate goal of creating an automated controller capable of providing real-time feedback and calibration of the Spill Regulation System (SRS) parameters on a millisecond timescale. We treat the Mu2e accelerator system as a Markov Decision Process suitable for Reinforcement Learning (RL), utilizing PPO to reduce bias and enhance training stability. A key innovation in our approach is the integration of a neuralized Proportional-Integral-Derivative (PID) controller into the policy function, resulting in a significant improvement in the Spill Duty Factor (SDF) by 13.6%, surpassing the performance of the current PID controller baseline by an additional 1.6%. This paper presents the preliminary offline results based on a differentiable simulator of the Mu2e accelerator. It paves the groundwork for real-time implementations and applications, representing a crucial step towards automated proton beam intensity control for the Mu2e experiment. |
| title | Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e |
| topic | Machine Learning Artificial Intelligence Accelerator Physics |
| url | https://arxiv.org/abs/2312.17372 |