Evaluation of Reinforcement Learning for Autonomous Penetration Testing using A3C, Q-learning and DQN
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917730401124352 |
|---|---|
| author | Becker, Norman Reti, Daniel Ntagiou, Evridiki V. Wallum, Marcus Schotten, Hans D. |
| author_facet | Becker, Norman Reti, Daniel Ntagiou, Evridiki V. Wallum, Marcus Schotten, Hans D. |
| contents | Penetration testing is the process of searching for security weaknesses by simulating an attack. It is usually performed by experienced professionals, where scanning and attack tools are applied. By automating the execution of such tools, the need for human interaction and decision-making could be reduced. In this work, a Network Attack Simulator (NASim) was used as an environment to train reinforcement learning agents to solve three predefined security scenarios. These scenarios cover techniques of exploitation, post-exploitation and wiretapping. A large hyperparameter grid search was performed to find the best hyperparameter combinations. The algorithms Q-learning, DQN and A3C were used, whereby A3C was able to solve all scenarios and achieve generalization. In addition, A3C could solve these scenarios with fewer actions than the baseline automated penetration testing. Although the training was performed on rather small scenarios and with small state and action spaces for the agents, the results show that a penetration test can successfully be performed by the RL agent. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_15656 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Evaluation of Reinforcement Learning for Autonomous Penetration Testing using A3C, Q-learning and DQN Becker, Norman Reti, Daniel Ntagiou, Evridiki V. Wallum, Marcus Schotten, Hans D. Cryptography and Security Artificial Intelligence Penetration testing is the process of searching for security weaknesses by simulating an attack. It is usually performed by experienced professionals, where scanning and attack tools are applied. By automating the execution of such tools, the need for human interaction and decision-making could be reduced. In this work, a Network Attack Simulator (NASim) was used as an environment to train reinforcement learning agents to solve three predefined security scenarios. These scenarios cover techniques of exploitation, post-exploitation and wiretapping. A large hyperparameter grid search was performed to find the best hyperparameter combinations. The algorithms Q-learning, DQN and A3C were used, whereby A3C was able to solve all scenarios and achieve generalization. In addition, A3C could solve these scenarios with fewer actions than the baseline automated penetration testing. Although the training was performed on rather small scenarios and with small state and action spaces for the agents, the results show that a penetration test can successfully be performed by the RL agent. |
| title | Evaluation of Reinforcement Learning for Autonomous Penetration Testing using A3C, Q-learning and DQN |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2407.15656 |