Structure-Enhanced Deep Reinforcement Learning for Optimal Transmission Scheduling
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929559659610112 |
|---|---|
| author | Chen, Jiazheng Liu, Wanchun Quevedo, Daniel E. Li, Yonghui Vucetic, Branka |
| author_facet | Chen, Jiazheng Liu, Wanchun Quevedo, Daniel E. Li, Yonghui Vucetic, Branka |
| contents | Remote state estimation of large-scale distributed dynamic processes plays an important role in Industry 4.0 applications. In this paper, by leveraging the theoretical results of structural properties of optimal scheduling policies, we develop a structure-enhanced deep reinforcement learning (DRL) framework for optimal scheduling of a multi-sensor remote estimation system to achieve the minimum overall estimation mean-square error (MSE). In particular, we propose a structure-enhanced action selection method, which tends to select actions that obey the policy structure. This explores the action space more effectively and enhances the learning efficiency of DRL agents. Furthermore, we introduce a structure-enhanced loss function to add penalty to actions that do not follow the policy structure. The new loss function guides the DRL to converge to the optimal policy structure quickly. Our numerical results show that the proposed structure-enhanced DRL algorithms can save the training time by 50% and reduce the remote estimation MSE by 10% to 25%, when compared to benchmark DRL algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2211_10827 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Structure-Enhanced Deep Reinforcement Learning for Optimal Transmission Scheduling Chen, Jiazheng Liu, Wanchun Quevedo, Daniel E. Li, Yonghui Vucetic, Branka Information Theory Artificial Intelligence Machine Learning Systems and Control Signal Processing Remote state estimation of large-scale distributed dynamic processes plays an important role in Industry 4.0 applications. In this paper, by leveraging the theoretical results of structural properties of optimal scheduling policies, we develop a structure-enhanced deep reinforcement learning (DRL) framework for optimal scheduling of a multi-sensor remote estimation system to achieve the minimum overall estimation mean-square error (MSE). In particular, we propose a structure-enhanced action selection method, which tends to select actions that obey the policy structure. This explores the action space more effectively and enhances the learning efficiency of DRL agents. Furthermore, we introduce a structure-enhanced loss function to add penalty to actions that do not follow the policy structure. The new loss function guides the DRL to converge to the optimal policy structure quickly. Our numerical results show that the proposed structure-enhanced DRL algorithms can save the training time by 50% and reduce the remote estimation MSE by 10% to 25%, when compared to benchmark DRL algorithms. |
| title | Structure-Enhanced Deep Reinforcement Learning for Optimal Transmission Scheduling |
| topic | Information Theory Artificial Intelligence Machine Learning Systems and Control Signal Processing |
| url | https://arxiv.org/abs/2211.10827 |