Soft Actor-Critic with Backstepping-Pretrained DeepONet for control of PDEs
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908880610525184 |
|---|---|
| author | Wang, Chenchen Qi, Jie Hu, Jiaqi |
| author_facet | Wang, Chenchen Qi, Jie Hu, Jiaqi |
| contents | This paper develops a reinforcement learning-based controller for the stabilization of partial differential equation (PDE) systems. Within the soft actor-critic (SAC) framework, we embed a DeepONet, a well-known neural operator (NO), which is pretrained using the backstepping controller. The pretrained DeepONet captures the essential features of the backstepping controller and serves as a feature extractor, replacing the convolutional neural networks (CNNs) layers in the original actor and critic networks, and directly connects to the fully connected layers of the SAC architecture. We apply this novel backstepping and reinforcement learning integrated method to stabilize an unstable ffrst-order hyperbolic PDE and an unstable reactiondiffusion PDE. Simulation results demonstrate that the proposed method outperforms the standard SAC, SAC with an untrained DeepONet, and the backstepping controller on both systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_04232 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Soft Actor-Critic with Backstepping-Pretrained DeepONet for control of PDEs Wang, Chenchen Qi, Jie Hu, Jiaqi Optimization and Control This paper develops a reinforcement learning-based controller for the stabilization of partial differential equation (PDE) systems. Within the soft actor-critic (SAC) framework, we embed a DeepONet, a well-known neural operator (NO), which is pretrained using the backstepping controller. The pretrained DeepONet captures the essential features of the backstepping controller and serves as a feature extractor, replacing the convolutional neural networks (CNNs) layers in the original actor and critic networks, and directly connects to the fully connected layers of the SAC architecture. We apply this novel backstepping and reinforcement learning integrated method to stabilize an unstable ffrst-order hyperbolic PDE and an unstable reactiondiffusion PDE. Simulation results demonstrate that the proposed method outperforms the standard SAC, SAC with an untrained DeepONet, and the backstepping controller on both systems. |
| title | Soft Actor-Critic with Backstepping-Pretrained DeepONet for control of PDEs |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2507.04232 |