Control-Informed Reinforcement Learning for Chemical Processes
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929476460347392 |
|---|---|
| author | Bloor, Maximilian Ahmed, Akhil Kotecha, Niki Mercangöz, Mehmet Tsay, Calvin Chanona, Ehecactl Antonio Del Rio |
| author_facet | Bloor, Maximilian Ahmed, Akhil Kotecha, Niki Mercangöz, Mehmet Tsay, Calvin Chanona, Ehecactl Antonio Del Rio |
| contents | This work proposes a control-informed reinforcement learning (CIRL) framework that integrates proportional-integral-derivative (PID) control components into the architecture of deep reinforcement learning (RL) policies. The proposed approach augments deep RL agents with a PID controller layer, incorporating prior knowledge from control theory into the learning process. CIRL improves performance and robustness by combining the best of both worlds: the disturbance-rejection and setpoint-tracking capabilities of PID control and the nonlinear modeling capacity of deep RL. Simulation studies conducted on a continuously stirred tank reactor system demonstrate the improved performance of CIRL compared to both conventional model-free deep RL and static PID controllers. CIRL exhibits better setpoint-tracking ability, particularly when generalizing to trajectories outside the training distribution, suggesting enhanced generalization capabilities. Furthermore, the embedded prior control knowledge within the CIRL policy improves its robustness to unobserved system disturbances. The control-informed RL framework combines the strengths of classical control and reinforcement learning to develop sample-efficient and robust deep reinforcement learning algorithms, with potential applications in complex industrial systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_13566 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Control-Informed Reinforcement Learning for Chemical Processes Bloor, Maximilian Ahmed, Akhil Kotecha, Niki Mercangöz, Mehmet Tsay, Calvin Chanona, Ehecactl Antonio Del Rio Systems and Control This work proposes a control-informed reinforcement learning (CIRL) framework that integrates proportional-integral-derivative (PID) control components into the architecture of deep reinforcement learning (RL) policies. The proposed approach augments deep RL agents with a PID controller layer, incorporating prior knowledge from control theory into the learning process. CIRL improves performance and robustness by combining the best of both worlds: the disturbance-rejection and setpoint-tracking capabilities of PID control and the nonlinear modeling capacity of deep RL. Simulation studies conducted on a continuously stirred tank reactor system demonstrate the improved performance of CIRL compared to both conventional model-free deep RL and static PID controllers. CIRL exhibits better setpoint-tracking ability, particularly when generalizing to trajectories outside the training distribution, suggesting enhanced generalization capabilities. Furthermore, the embedded prior control knowledge within the CIRL policy improves its robustness to unobserved system disturbances. The control-informed RL framework combines the strengths of classical control and reinforcement learning to develop sample-efficient and robust deep reinforcement learning algorithms, with potential applications in complex industrial systems. |
| title | Control-Informed Reinforcement Learning for Chemical Processes |
| topic | Systems and Control |
| url | https://arxiv.org/abs/2408.13566 |