Improving a Proportional Integral Controller with Reinforcement Learning on a Throttle Valve Benchmark
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917724151611392 |
|---|---|
| author | Daoudi, Paul Mavkov, Bojan Robu, Bogdan Prieur, Christophe Witrant, Emmanuel Barlier, Merwan Santos, Ludovic Dos |
| author_facet | Daoudi, Paul Mavkov, Bojan Robu, Bogdan Prieur, Christophe Witrant, Emmanuel Barlier, Merwan Santos, Ludovic Dos |
| contents | This paper presents a learning-based control strategy for non-linear throttle valves with an asymmetric hysteresis, leading to a near-optimal controller without requiring any prior knowledge about the environment. We start with a carefully tuned Proportional Integrator (PI) controller and exploit the recent advances in Reinforcement Learning (RL) with Guides to improve the closed-loop behavior by learning from the additional interactions with the valve. We test the proposed control method in various scenarios on three different valves, all highlighting the benefits of combining both PI and RL frameworks to improve control performance in non-linear stochastic systems. In all the experimental test cases, the resulting agent has a better sample efficiency than traditional RL agents and outperforms the PI controller. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_13654 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improving a Proportional Integral Controller with Reinforcement Learning on a Throttle Valve Benchmark Daoudi, Paul Mavkov, Bojan Robu, Bogdan Prieur, Christophe Witrant, Emmanuel Barlier, Merwan Santos, Ludovic Dos Systems and Control Machine Learning This paper presents a learning-based control strategy for non-linear throttle valves with an asymmetric hysteresis, leading to a near-optimal controller without requiring any prior knowledge about the environment. We start with a carefully tuned Proportional Integrator (PI) controller and exploit the recent advances in Reinforcement Learning (RL) with Guides to improve the closed-loop behavior by learning from the additional interactions with the valve. We test the proposed control method in various scenarios on three different valves, all highlighting the benefits of combining both PI and RL frameworks to improve control performance in non-linear stochastic systems. In all the experimental test cases, the resulting agent has a better sample efficiency than traditional RL agents and outperforms the PI controller. |
| title | Improving a Proportional Integral Controller with Reinforcement Learning on a Throttle Valve Benchmark |
| topic | Systems and Control Machine Learning |
| url | https://arxiv.org/abs/2402.13654 |