Rethinking Adversarial Attacks in Reinforcement Learning from Policy Distribution Perspective
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916555749588992 |
|---|---|
| author | Duan, Tianyang Zhang, Zongyuan Lin, Zheng Gao, Yue Xiong, Ling Cui, Yong Liang, Hongbin Chen, Xianhao Cui, Heming Huang, Dong |
| author_facet | Duan, Tianyang Zhang, Zongyuan Lin, Zheng Gao, Yue Xiong, Ling Cui, Yong Liang, Hongbin Chen, Xianhao Cui, Heming Huang, Dong |
| contents | Deep Reinforcement Learning (DRL) suffers from uncertainties and inaccuracies in the observation signal in realworld applications. Adversarial attack is an effective method for evaluating the robustness of DRL agents. However, existing attack methods targeting individual sampled actions have limited impacts on the overall policy distribution, particularly in continuous action spaces. To address these limitations, we propose the Distribution-Aware Projected Gradient Descent attack (DAPGD). DAPGD uses distribution similarity as the gradient perturbation input to attack the policy network, which leverages the entire policy distribution rather than relying on individual samples. We utilize the Bhattacharyya distance in DAPGD to measure policy similarity, enabling sensitive detection of subtle but critical differences between probability distributions. Our experiment results demonstrate that DAPGD achieves SOTA results compared to the baselines in three robot navigation tasks, achieving an average 22.03% higher reward drop compared to the best baseline. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_03562 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rethinking Adversarial Attacks in Reinforcement Learning from Policy Distribution Perspective Duan, Tianyang Zhang, Zongyuan Lin, Zheng Gao, Yue Xiong, Ling Cui, Yong Liang, Hongbin Chen, Xianhao Cui, Heming Huang, Dong Machine Learning Artificial Intelligence Deep Reinforcement Learning (DRL) suffers from uncertainties and inaccuracies in the observation signal in realworld applications. Adversarial attack is an effective method for evaluating the robustness of DRL agents. However, existing attack methods targeting individual sampled actions have limited impacts on the overall policy distribution, particularly in continuous action spaces. To address these limitations, we propose the Distribution-Aware Projected Gradient Descent attack (DAPGD). DAPGD uses distribution similarity as the gradient perturbation input to attack the policy network, which leverages the entire policy distribution rather than relying on individual samples. We utilize the Bhattacharyya distance in DAPGD to measure policy similarity, enabling sensitive detection of subtle but critical differences between probability distributions. Our experiment results demonstrate that DAPGD achieves SOTA results compared to the baselines in three robot navigation tasks, achieving an average 22.03% higher reward drop compared to the best baseline. |
| title | Rethinking Adversarial Attacks in Reinforcement Learning from Policy Distribution Perspective |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2501.03562 |