State-Aware Perturbation Optimization for Robust Deep Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917968487645184 |
|---|---|
| author | Zhang, Zongyuan Duan, Tianyang Lin, Zheng Huang, Dong Fang, Zihan Sun, Zekai Xiong, Ling Liang, Hongbin Cui, Heming Cui, Yong |
| author_facet | Zhang, Zongyuan Duan, Tianyang Lin, Zheng Huang, Dong Fang, Zihan Sun, Zekai Xiong, Ling Liang, Hongbin Cui, Heming Cui, Yong |
| contents | Recently, deep reinforcement learning (DRL) has emerged as a promising approach for robotic control. However, the deployment of DRL in real-world robots is hindered by its sensitivity to environmental perturbations. While existing whitebox adversarial attacks rely on local gradient information and apply uniform perturbations across all states to evaluate DRL robustness, they fail to account for temporal dynamics and state-specific vulnerabilities. To combat the above challenge, we first conduct a theoretical analysis of white-box attacks in DRL by establishing the adversarial victim-dynamics Markov decision process (AVD-MDP), to derive the necessary and sufficient conditions for a successful attack. Based on this, we propose a selective state-aware reinforcement adversarial attack method, named STAR, to optimize perturbation stealthiness and state visitation dispersion. STAR first employs a soft mask-based state-targeting mechanism to minimize redundant perturbations, enhancing stealthiness and attack effectiveness. Then, it incorporates an information-theoretic optimization objective to maximize mutual information between perturbations, environmental states, and victim actions, ensuring a dispersed state-visitation distribution that steers the victim agent into vulnerable states for maximum return reduction. Extensive experiments demonstrate that STAR outperforms state-of-the-art benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_20613 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | State-Aware Perturbation Optimization for Robust Deep Reinforcement Learning Zhang, Zongyuan Duan, Tianyang Lin, Zheng Huang, Dong Fang, Zihan Sun, Zekai Xiong, Ling Liang, Hongbin Cui, Heming Cui, Yong Machine Learning Artificial Intelligence Networking and Internet Architecture Systems and Control Recently, deep reinforcement learning (DRL) has emerged as a promising approach for robotic control. However, the deployment of DRL in real-world robots is hindered by its sensitivity to environmental perturbations. While existing whitebox adversarial attacks rely on local gradient information and apply uniform perturbations across all states to evaluate DRL robustness, they fail to account for temporal dynamics and state-specific vulnerabilities. To combat the above challenge, we first conduct a theoretical analysis of white-box attacks in DRL by establishing the adversarial victim-dynamics Markov decision process (AVD-MDP), to derive the necessary and sufficient conditions for a successful attack. Based on this, we propose a selective state-aware reinforcement adversarial attack method, named STAR, to optimize perturbation stealthiness and state visitation dispersion. STAR first employs a soft mask-based state-targeting mechanism to minimize redundant perturbations, enhancing stealthiness and attack effectiveness. Then, it incorporates an information-theoretic optimization objective to maximize mutual information between perturbations, environmental states, and victim actions, ensuring a dispersed state-visitation distribution that steers the victim agent into vulnerable states for maximum return reduction. Extensive experiments demonstrate that STAR outperforms state-of-the-art benchmarks. |
| title | State-Aware Perturbation Optimization for Robust Deep Reinforcement Learning |
| topic | Machine Learning Artificial Intelligence Networking and Internet Architecture Systems and Control |
| url | https://arxiv.org/abs/2503.20613 |